-- Logs begin at Wed 2022-09-14 01:27:11 UTC, end at Wed 2022-09-14 02:04:32 UTC. -- Sep 14 01:52:29.923489 dv-cinder-857075-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 14 01:52:29.932023 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 14 01:52:29.934076 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Sep 14 01:52:29 2022] *** Sep 14 01:52:29.934076 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 14 01:52:29.934076 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 14 01:52:29.934076 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: nodename: dv-cinder-857075-5-iscsi Sep 14 01:52:29.934076 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: machine: x86_64 Sep 14 01:52:29.934076 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: clock source: unix Sep 14 01:52:29.934076 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: pcre jit disabled Sep 14 01:52:29.934076 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: detected number of CPU cores: 4 Sep 14 01:52:29.934076 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: current working directory: / Sep 14 01:52:29.934076 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: detected binary path: /usr/bin/uwsgi-core Sep 14 01:52:29.934740 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: your processes number limit is 47680 Sep 14 01:52:29.934740 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: your memory page size is 4096 bytes Sep 14 01:52:29.934740 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: detected max file descriptor number: 2048 Sep 14 01:52:29.934740 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: lock engine: pthread robust mutexes Sep 14 01:52:29.934740 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: thunder lock: enabled Sep 14 01:52:29.934740 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 14 01:52:29.934740 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 14 01:52:29.972871 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: Python main interpreter initialized at 0x564326b699c0 Sep 14 01:52:29.972871 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: python threads support enabled Sep 14 01:52:29.972871 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: your server socket listen backlog is limited to 100 connections Sep 14 01:52:29.972871 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: your mercy for graceful operations on workers is 90 seconds Sep 14 01:52:29.973221 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: mapped 403077 bytes (393 KB) for 2 cores Sep 14 01:52:29.973304 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: *** Operational MODE: preforking *** Sep 14 01:52:29.973549 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: *** uWSGI is running in multiple interpreter mode *** Sep 14 01:52:29.973549 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: spawned uWSGI master process (pid: 81370) Sep 14 01:52:29.973745 dv-cinder-857075-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 14 01:52:29.974733 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: spawned uWSGI worker 1 (pid: 81382, cores: 1) Sep 14 01:52:29.974733 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: spawned uWSGI worker 2 (pid: 81383, cores: 1) Sep 14 01:52:29.974904 dv-cinder-857075-5-iscsi devstack@keystone.service[81370]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 14 01:52:34.215615 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 14 01:52:34.215615 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: from cryptography.utils import int_from_bytes Sep 14 01:52:34.216410 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 14 01:52:34.216410 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: from cryptography.utils import int_from_bytes Sep 14 01:52:34.216831 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 14 01:52:34.216831 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: from cryptography.utils import int_from_bytes Sep 14 01:52:34.216943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 14 01:52:34.216943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: from cryptography.utils import int_from_bytes Sep 14 01:52:34.266474 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 14 01:52:34.266631 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 14 01:52:34.266720 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 14 01:52:34.266804 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 14 01:52:34.266883 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] command line args: [] {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 14 01:52:34.266960 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] command line args: [] {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 14 01:52:34.267100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 14 01:52:34.267179 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 14 01:52:34.267278 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ================================================================================ {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 14 01:52:34.267363 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ================================================================================ {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 14 01:52:34.267531 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] admin_token = **** {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.267608 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] admin_token = **** {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.267775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] config_dir = [] {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.267859 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] config_dir = [] {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.267961 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.268045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.268147 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] config_source = [] {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.268230 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] config_source = [] {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.268313 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] debug = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.268391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] debug = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.268507 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.269755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.269755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.269906 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.269972 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.270082 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.270149 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.270266 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.270357 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.270437 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.270515 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] list_limit = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.270592 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] list_limit = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.270664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] log_config_append = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.270739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] log_config_append = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.270819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.270896 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.270896 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] log_dir = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.271024 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] log_dir = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.271090 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] log_file = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.271199 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] log_file = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.271288 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.271367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.271446 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.271519 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.271590 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.271661 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.271738 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.271807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.271884 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.271961 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.272032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.272106 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.272177 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.272247 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.272317 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.272383 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.272464 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.272545 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.272619 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.272689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.272764 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.272841 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.272995 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.273088 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.273243 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.273339 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.273485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.273562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.273759 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.273845 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.274271 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.274440 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.274440 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] publish_errors = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.274634 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] publish_errors = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.274709 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.274830 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.274905 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.275029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.275106 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.275225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.275301 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.275429 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.275513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.276225 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.276307 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] standard_threads = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.276379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] standard_threads = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.276535 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.276633 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.276765 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.276845 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.276943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.277027 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.277100 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] use_journal = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.277223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] use_journal = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.277303 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] use_json = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.277375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] use_json = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.277451 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] use_stderr = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.277528 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] use_stderr = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.277600 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] use_syslog = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.277678 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] use_syslog = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.277818 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.277903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:52:34.278099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.278188 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.278338 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.278426 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.278596 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.278679 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.278836 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.278914 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.279082 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.279170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.279318 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.279396 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.279563 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.279645 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.279799 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.279885 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.280046 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.280124 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.280291 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.280368 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.280538 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.280624 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.280771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.280890 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.281022 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.281150 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.281255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.281346 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.281500 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.281585 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.281743 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.281828 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.281994 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.282087 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.282250 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.282328 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.282502 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.282579 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.282728 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.282803 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.282970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.283055 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.283218 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.283295 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.283475 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.283553 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.283726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.283802 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.283973 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.284051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.284204 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.284282 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.284434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.284510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.284661 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.284737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.284896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.284974 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.285144 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.285221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.285390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.285473 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.285618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.285701 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.285847 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.285924 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.286115 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.286194 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.286369 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.286449 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.286607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.286685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.286854 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.286932 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.287087 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.287165 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.287323 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.287401 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.287556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.287632 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.287784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.287861 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.288013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.288090 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.288240 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.288326 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.288488 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.288568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.288693 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.288776 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.288937 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.289014 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.289167 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.289246 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.289401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.289484 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.289634 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.289711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.289867 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.289945 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.290124 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.290209 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.290375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.290465 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.290607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.290691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.290840 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.290924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.291075 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.291206 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.291346 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.291426 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.291582 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.291706 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.291855 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.292073 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.292073 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.292469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.292469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.292797 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.292981 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.293200 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.293376 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.293555 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.293710 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.293940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.294192 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.294418 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.294639 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.294854 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.295019 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.295209 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.295401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.295590 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.295814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.296019 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.296183 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.296374 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.296535 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.296666 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.296826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.296963 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.297130 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.297273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.297401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.297534 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.297662 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.297830 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.297964 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.298123 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.298251 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.298378 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.298506 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.298648 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.298648 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.298891 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.299025 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.299193 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.299327 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.299464 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.299593 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.299721 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.299849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.299976 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.300104 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.300239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.300367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.300494 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.300622 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.300749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.300749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.301020 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.301150 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.301278 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.301406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.301533 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.301717 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.301918 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.302080 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.302211 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.302346 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.302475 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.302602 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.302807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.302948 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.303077 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.303204 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.303331 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.303468 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.303605 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.303742 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.303876 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.304011 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.304141 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.304268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.304404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.304547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.304712 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.304856 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.304984 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.305113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.305240 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.305376 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.305504 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.305640 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.305768 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.305896 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.306039 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.306185 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.306315 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.306442 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.306576 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.306711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.306872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.307013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] auth.external = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.307142 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] auth.external = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.307280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.307410 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.307538 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.307667 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.307796 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.307924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.308064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] auth.password = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.308194 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] auth.password = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.308324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] auth.token = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.308469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] auth.token = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.308598 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.308726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.308862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.308991 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.309119 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.309247 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.309385 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.309513 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.309649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.309785 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.309915 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.310086 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.310228 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.310358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.310496 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] credential.caching = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.310626 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] credential.caching = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.310754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.310882 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.311020 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.311150 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.311288 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.311417 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.311546 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.311682 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.311810 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.311938 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.312087 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.312237 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.312371 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.312508 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.312637 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.312766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.312902 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.313032 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.313161 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.313289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.313417 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.313554 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.313684 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.313812 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.313950 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.314104 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.314234 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.314373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.314502 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.314632 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.314795 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] federation.caching = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.314929 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] federation.caching = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.315058 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.315194 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.315324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.315452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.315581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.315723 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.315853 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.315989 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.316119 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.316255 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.316385 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.316513 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.316658 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.316797 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.316928 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.317057 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.317185 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.317313 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.317451 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.317588 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.317718 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.317847 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.318001 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] identity.caching = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.318151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] identity.caching = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.318281 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.318490 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.318622 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.318753 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.318882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.319011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.319140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.319276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.319422 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.319551 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.319680 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.319809 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.319937 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.320077 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.320215 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.320344 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.320473 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.320601 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.320730 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.320875 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.321018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.321148 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.321288 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.321417 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.321551 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.321689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.321818 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.321947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.322155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.322285 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.322909 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.323058 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.323188 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.323317 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.323446 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.323584 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.323722 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.323851 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.323981 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.324108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.324242 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.324459 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.324596 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.324726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.324890 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.325024 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.325154 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.325282 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.325419 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.325558 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.325688 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.325817 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.325946 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.326105 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.326248 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.326378 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.326507 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.326635 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.326770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.326899 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.327036 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.327167 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.327307 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.327436 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.327565 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.327693 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.327830 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.327959 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.328088 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.328218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.328347 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.328475 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.328611 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.328741 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.328870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.328998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.329136 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.329266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.329403 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.329532 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.329661 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.329798 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.329928 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.330078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.330219 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.330351 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.330480 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.330608 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.330736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.330871 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.331027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.331158 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.331286 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.331422 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.331551 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.331679 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.331815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.331945 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.332073 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.332201 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.332329 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.user = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.332458 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.user = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.332600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.332740 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.332869 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.332997 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.333126 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.333254 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.333390 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.333520 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.333649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.333776 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.333904 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.334078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.334224 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.334354 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.334483 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.334621 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.334751 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.334914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.335060 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.335190 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.335318 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.335453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.335590 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.335719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.335855 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.335985 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.336119 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.336248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.336386 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.336514 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.336651 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.336780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.336908 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.337036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.337164 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.337292 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.337429 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.337558 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.337686 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.337814 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.337942 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.338124 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.338791 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.338923 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.339052 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.339180 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.339308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.339436 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.339572 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.339702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.339830 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.339958 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.340086 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.340213 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.340349 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.340478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.340615 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.340744 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.340872 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.341008 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.341148 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.341278 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.341406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.341534 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.341662 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.341799 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.341929 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.342088 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.342219 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.342348 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.342485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.342625 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.342755 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] resource.caching = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.342884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] resource.caching = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.343012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.343139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.343267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.343403 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.343533 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.343661 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.343789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.343918 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.344053 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.344200 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.344329 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.344457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.344585 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.344720 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.344855 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.345101 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.345244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.345379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.345514 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] role.caching = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.345648 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] role.caching = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.345783 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] role.driver = sql {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.345925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] role.driver = sql {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.346091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.346238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.346374 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.346509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.346643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.346790 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.346927 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.347063 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.347198 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.347333 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.347467 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.347610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.347747 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.347882 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.348030 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.348166 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.348300 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.348443 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.348578 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.348712 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.348846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.348980 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.349114 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.349262 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.349399 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.349533 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.349668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.349816 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.349950 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.350155 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.350294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.350429 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.350563 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.350698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.350841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.350983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.351122 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.351259 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.351393 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.351528 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.351671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.351817 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.351953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.352088 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.352223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.352359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.352502 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.352637 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.352771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.352906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.353040 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.353174 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.353316 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.353453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.353597 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.353771 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.353909 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.354064 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.354213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.354350 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.354485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.354620 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.354754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.354887 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.355063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.355208 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.355343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] token.caching = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.355489 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] token.caching = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.355623 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.355758 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.355901 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.356036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.356204 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.356344 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.356479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.356614 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.356757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.356893 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.357027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.357161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.357322 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.357474 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.357620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.357756 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.357890 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.358041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.358183 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.358327 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.358473 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.358609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.358744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.358879 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.359014 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.359148 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.359291 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.359437 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.359572 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.359717 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.359866 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.360009 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.360146 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.360281 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.360415 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] database.connection = **** {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.360558 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] database.connection = **** {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.360695 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.360830 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.360965 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.361111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.361247 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.361400 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.361537 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.361672 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.361808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.361943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.362135 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.362288 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.362426 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.362561 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.362695 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.362832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.362967 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.363110 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.363260 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.363396 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.363531 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.363665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.363800 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.363943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.364079 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.364215 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.364349 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.364483 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.364617 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.364759 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.364897 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.365031 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.365214 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.365355 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.365499 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.365636 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:52:34.365771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 14 01:52:34.365908 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 14 01:52:34.382279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.382493 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.382719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.382874 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.383107 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.383336 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.391455 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.393686 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.420465 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.420590 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.420729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.420835 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.473914 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.474113 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.523262 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.523493 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.527291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.527529 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.678234 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.678462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.678667 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.678884 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.678991 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.679100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.679183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.679315 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.679398 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.679536 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.679755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.679973 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.702595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.702825 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.702909 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.703050 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.703134 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.703276 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.703358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.703496 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.703589 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.703713 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.703796 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.703929 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.704009 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.704147 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.715553 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.715916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.728106 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.728284 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.728507 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.728591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.728751 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.728841 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.728969 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.729052 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.729184 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.729268 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.729397 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.729479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.729652 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.729872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:52:34.746041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.746254 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 14 01:52:34.746598 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.746801 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 14 01:52:34.747078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.747340 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.747599 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.747857 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.750634 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.750848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 14 01:52:34.751197 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.751402 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 14 01:52:34.751771 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.752245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.752722 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.753200 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.757425 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.757701 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.757975 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.758287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.765068 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.766038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.767606 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.768918 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.769207 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.769556 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.769775 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.770025 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.770110 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.770327 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.770410 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.770599 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.770876 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.771148 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.771421 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.771689 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.778193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.781066 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.781355 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.781640 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.781924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.782233 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.782497 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.782769 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.783045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.783322 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.783595 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.783860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.794697 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.796179 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.796468 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.801156 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.801451 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.801736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.806532 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.808555 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.808648 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.808842 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.808939 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.813446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.813738 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.814037 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.814545 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.814847 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.815140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.815427 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.815711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.815992 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.816267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.816540 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.820286 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.820572 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.826248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.826547 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.826834 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.827114 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.827388 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.827660 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.827929 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.828195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.831777 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.832059 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.835867 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.839089 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.839387 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.843432 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.843529 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.843712 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.846676 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.847521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.849882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.850208 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.850498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.850756 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.850841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.851052 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.851137 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.851343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.855730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.858818 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.862072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.862372 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.862657 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.862942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.863220 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.863508 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.867414 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.867708 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.867981 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.868250 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.875156 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.876695 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.878247 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.878527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.879053 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.879342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.879612 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.879877 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.881148 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.881435 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.881717 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.882024 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.882427 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.887428 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.888958 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.890501 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.890777 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.893472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.893765 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.894072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.894245 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.894358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.894535 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.894752 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.894835 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.895088 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.895468 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.895758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.896047 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.905742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.906046 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.906329 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.906611 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.907001 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.907283 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.907570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.911316 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.914242 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.917788 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.919254 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.920723 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.921011 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.921410 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.923179 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.926080 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.927758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.928939 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.929217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.930464 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.931987 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.932165 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.932272 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.932567 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.932660 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.932858 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.933142 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.938941 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.940387 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.940860 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.941160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.941444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.943267 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.943661 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.943947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.944227 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.948801 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.949089 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.949373 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.949649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.949931 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.950224 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.950499 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.950775 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.951056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.951342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.951633 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.951894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.952160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.952427 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.952508 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.952755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.953045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.959837 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:52:34.960122 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.960404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.960680 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.960952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.961230 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.961505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.961780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.962070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.962360 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.962638 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.962905 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.963226 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.963506 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:52:34.987105 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x564326b699c0 pid: 81383 (default app) Sep 14 01:52:34.990741 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73ce5a-e60c-41f4-af4f-bc31e091a566 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:34.991028 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73ce5a-e60c-41f4-af4f-bc31e091a566 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:34.991346 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73ce5a-e60c-41f4-af4f-bc31e091a566 None None] PATH_INFO: `/v3` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:34.996453 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Wed Sep 14 01:52:34 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:52:35.000281 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x564326b699c0 pid: 81382 (default app) Sep 14 01:52:41.791927 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e317ae8-5e2d-42a9-8c69-5610d257f9f6 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:41.792508 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e317ae8-5e2d-42a9-8c69-5610d257f9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:41.792508 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e317ae8-5e2d-42a9-8c69-5610d257f9f6 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:41.793704 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 2/2] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:52:41.801232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a7752-4e70-4a1d-824e-e6b7c33b7a79 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:41.801759 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a7752-4e70-4a1d-824e-e6b7c33b7a79 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:41.801759 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a7752-4e70-4a1d-824e-e6b7c33b7a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:41.871634 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: INFO dbcounter [None req-6c5a7752-4e70-4a1d-824e-e6b7c33b7a79 None None] Registered counter for database keystone Sep 14 01:52:41.890697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG oslo_db.sqlalchemy.engines [None req-6c5a7752-4e70-4a1d-824e-e6b7c33b7a79 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81382) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 14 01:52:41.896800 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writer thread running {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 14 01:52:42.087577 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG passlib.handlers.bcrypt [None req-6c5a7752-4e70-4a1d-824e-e6b7c33b7a79 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81382) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 14 01:52:42.087964 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG passlib.handlers.bcrypt [None req-6c5a7752-4e70-4a1d-824e-e6b7c33b7a79 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81382) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 14 01:52:42.146128 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-6c5a7752-4e70-4a1d-824e-e6b7c33b7a79 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:52:42.147707 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-6c5a7752-4e70-4a1d-824e-e6b7c33b7a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:52:42.272137 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 1/3] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:52:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 14 01:52:42.735549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e317ae8-5e2d-42a9-8c69-5610d257f9f6 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:42.804895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: INFO dbcounter [None req-7e317ae8-5e2d-42a9-8c69-5610d257f9f6 None None] Registered counter for database keystone Sep 14 01:52:42.826880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG oslo_db.sqlalchemy.engines [None req-7e317ae8-5e2d-42a9-8c69-5610d257f9f6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81383) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 14 01:52:42.833005 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writer thread running {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 14 01:52:42.952577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e317ae8-5e2d-42a9-8c69-5610d257f9f6 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:42.956810 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95688680-9bbe-41c6-a65e-8bffaae92c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ca72c26a2124532970b02d36e64c3f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:42.957425 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95688680-9bbe-41c6-a65e-8bffaae92c3f admin admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:42.957703 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95688680-9bbe-41c6-a65e-8bffaae92c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:42.957973 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95688680-9bbe-41c6-a65e-8bffaae92c3f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:42.966174 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95688680-9bbe-41c6-a65e-8bffaae92c3f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:43.018746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: WARNING keystone.common.rbac_enforcer.enforcer [None req-95688680-9bbe-41c6-a65e-8bffaae92c3f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 14 01:52:43.019031 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95688680-9bbe-41c6-a65e-8bffaae92c3f admin admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:43.028408 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 3/4] 10.222.0.15 () {58 vars in 1324 bytes} [Wed Sep 14 01:52:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 01:52:43.034306 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a7752-4e70-4a1d-824e-e6b7c33b7a79 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:43.049253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a7752-4e70-4a1d-824e-e6b7c33b7a79 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:43.053610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadba520-3f79-4e62-b051-7e720a852e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ca72c26a2124532970b02d36e64c3f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:43.054736 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadba520-3f79-4e62-b051-7e720a852e21 admin admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:43.055019 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadba520-3f79-4e62-b051-7e720a852e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:43.055288 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadba520-3f79-4e62-b051-7e720a852e21 admin admin] PATH_INFO: `/v3/projects/2ca72c26a2124532970b02d36e64c3f9` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:43.057304 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadba520-3f79-4e62-b051-7e720a852e21 admin admin] RBAC: Authorizing `identity:get_project(project_id=2ca72c26a2124532970b02d36e64c3f9)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:43.110776 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: WARNING keystone.common.rbac_enforcer.enforcer [None req-eadba520-3f79-4e62-b051-7e720a852e21 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 14 01:52:43.111121 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadba520-3f79-4e62-b051-7e720a852e21 admin admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:43.115636 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 2/5] 10.222.0.15 () {56 vars in 1207 bytes} [Wed Sep 14 01:52:43 2022] GET /identity/v3/projects/2ca72c26a2124532970b02d36e64c3f9 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:52:45.229346 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87bdb3-4c73-4619-89b7-8b001c9bf46c None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:45.229884 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87bdb3-4c73-4619-89b7-8b001c9bf46c None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:45.229884 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87bdb3-4c73-4619-89b7-8b001c9bf46c None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:45.234136 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 4/6] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:52:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:52:45.239397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd358351-c490-42cf-9da5-ca65295b8f12 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:45.239808 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd358351-c490-42cf-9da5-ca65295b8f12 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:45.239944 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd358351-c490-42cf-9da5-ca65295b8f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:45.261046 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-dd358351-c490-42cf-9da5-ca65295b8f12 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:52:45.262705 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-dd358351-c490-42cf-9da5-ca65295b8f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:52:45.270546 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 3/7] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:52:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 14 01:52:45.535170 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87bdb3-4c73-4619-89b7-8b001c9bf46c None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:45.547652 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87bdb3-4c73-4619-89b7-8b001c9bf46c None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:45.551774 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c98c7-f334-4462-9d99-3d2eb1e660c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ca72c26a2124532970b02d36e64c3f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:45.552370 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c98c7-f334-4462-9d99-3d2eb1e660c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:45.552650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c98c7-f334-4462-9d99-3d2eb1e660c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:45.552920 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c98c7-f334-4462-9d99-3d2eb1e660c6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:45.560560 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c98c7-f334-4462-9d99-3d2eb1e660c6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:45.562345 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c98c7-f334-4462-9d99-3d2eb1e660c6 admin admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:45.570944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 5/8] 10.222.0.15 () {58 vars in 1324 bytes} [Wed Sep 14 01:52:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 14 01:52:45.576748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd358351-c490-42cf-9da5-ca65295b8f12 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:45.585667 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd358351-c490-42cf-9da5-ca65295b8f12 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:45.590526 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da3dfe-6c18-4350-b49c-d299e97168ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ca72c26a2124532970b02d36e64c3f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:45.591141 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da3dfe-6c18-4350-b49c-d299e97168ac admin admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:45.591417 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da3dfe-6c18-4350-b49c-d299e97168ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:45.591684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da3dfe-6c18-4350-b49c-d299e97168ac admin admin] PATH_INFO: `/v3/users/25374d78618f4e32be8780901a31679f` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:45.592973 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da3dfe-6c18-4350-b49c-d299e97168ac admin admin] RBAC: Authorizing `identity:get_user(user_id=25374d78618f4e32be8780901a31679f)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:45.594815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da3dfe-6c18-4350-b49c-d299e97168ac admin admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:45.596335 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 4/9] 10.222.0.15 () {56 vars in 1198 bytes} [Wed Sep 14 01:52:45 2022] GET /identity/v3/users/25374d78618f4e32be8780901a31679f => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:52:47.774157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b438b-50e6-47e1-b751-8416d7cc33f2 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:47.775233 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b438b-50e6-47e1-b751-8416d7cc33f2 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:47.775233 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b438b-50e6-47e1-b751-8416d7cc33f2 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:47.776561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 6/10] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:52:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:52:47.783211 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e4a08-b639-4bdd-8349-d450f3169e63 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:47.783668 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e4a08-b639-4bdd-8349-d450f3169e63 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:47.783763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e4a08-b639-4bdd-8349-d450f3169e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:47.806014 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-e79e4a08-b639-4bdd-8349-d450f3169e63 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:52:47.807191 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-e79e4a08-b639-4bdd-8349-d450f3169e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:52:47.821646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad9df4-76a0-4978-b663-6f89b613c495 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:47.821940 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad9df4-76a0-4978-b663-6f89b613c495 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:47.822236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad9df4-76a0-4978-b663-6f89b613c495 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:47.823556 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 7/11] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:52:47.828436 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f981eb-0526-4c96-97ef-71857e16768e None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:47.828721 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f981eb-0526-4c96-97ef-71857e16768e None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:47.828997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f981eb-0526-4c96-97ef-71857e16768e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:47.870504 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG passlib.handlers.bcrypt [None req-16f981eb-0526-4c96-97ef-71857e16768e None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81383) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 14 01:52:47.870890 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG passlib.handlers.bcrypt [None req-16f981eb-0526-4c96-97ef-71857e16768e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81383) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 14 01:52:47.886844 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 5/12] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:52:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:52:47.919849 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-16f981eb-0526-4c96-97ef-71857e16768e None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:52:47.921416 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-16f981eb-0526-4c96-97ef-71857e16768e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:52:48.019689 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 8/13] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:52:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:52:48.148577 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e4a08-b639-4bdd-8349-d450f3169e63 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:48.173631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:48.250734 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e4a08-b639-4bdd-8349-d450f3169e63 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:48.282738 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558241ec-7298-4da4-92e9-dc010c226979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:48.283372 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558241ec-7298-4da4-92e9-dc010c226979 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:48.283712 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558241ec-7298-4da4-92e9-dc010c226979 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:48.283929 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558241ec-7298-4da4-92e9-dc010c226979 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:48.285050 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f981eb-0526-4c96-97ef-71857e16768e None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:48.296141 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558241ec-7298-4da4-92e9-dc010c226979 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:52:48.304999 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-558241ec-7298-4da4-92e9-dc010c226979 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:52:48.304999 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:52:48.304999 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:52:48.304999 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:52:48.304999 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:52:48.304999 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:52:48.304999 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:52:48.304999 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:52:48.304999 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:52:48.304999 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:52:48.304999 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:52:48.304999 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:52:48.304999 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:52:48.304999 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:52:48.304999 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:52:48.304999 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:52:48.306585 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:52:48.306585 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:52:48.306585 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:52:48.306585 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:52:48.306585 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:52:48.306585 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:52:48.306585 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:52:48.306585 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:52:48.306585 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:52:48.306585 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:52:48.306585 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:52:48.306585 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:52:48.306585 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:52:48.306585 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:52:48.306585 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:52:48.306585 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:52:48.307704 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:52:48.307704 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:52:48.307704 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:52:48.307704 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:52:48.307704 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:52:48.307704 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:52:48.307704 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:52:48.307704 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:52:48.307704 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:52:48.307704 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:52:48.307704 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:52:48.307704 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:52:48.307704 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 6/14] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:52:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:52:48.312639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558241ec-7298-4da4-92e9-dc010c226979 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:48.384713 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f981eb-0526-4c96-97ef-71857e16768e None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:48.386936 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:48.399416 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:48.419368 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558241ec-7298-4da4-92e9-dc010c226979 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:48.422371 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1d297-432d-4ef0-8521-0db88a704144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:48.423228 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1d297-432d-4ef0-8521-0db88a704144 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:48.423512 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1d297-432d-4ef0-8521-0db88a704144 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:48.423785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1d297-432d-4ef0-8521-0db88a704144 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:48.453505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6edfc0-4f53-440c-abf1-4a3bb8941467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:48.454148 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6edfc0-4f53-440c-abf1-4a3bb8941467 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:48.454431 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6edfc0-4f53-440c-abf1-4a3bb8941467 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:48.454708 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6edfc0-4f53-440c-abf1-4a3bb8941467 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:48.455374 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6edfc0-4f53-440c-abf1-4a3bb8941467 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:52:48.455682 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6edfc0-4f53-440c-abf1-4a3bb8941467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:48.457083 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6edfc0-4f53-440c-abf1-4a3bb8941467 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:48.468543 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 7/15] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:52:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:52:48.474317 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6edfc0-4f53-440c-abf1-4a3bb8941467 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:48.489953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1d297-432d-4ef0-8521-0db88a704144 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:48.491509 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1d297-432d-4ef0-8521-0db88a704144 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:48.574892 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6edfc0-4f53-440c-abf1-4a3bb8941467 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:48.591209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 9/16] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:52:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:52:48.596955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1d297-432d-4ef0-8521-0db88a704144 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:48.599598 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:48.610562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a5b4c-4eda-431a-b8af-e9ff87dfa585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:48.611205 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a5b4c-4eda-431a-b8af-e9ff87dfa585 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:48.611481 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a5b4c-4eda-431a-b8af-e9ff87dfa585 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:48.612036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a5b4c-4eda-431a-b8af-e9ff87dfa585 None admin] PATH_INFO: `/v3/users/25374d78618f4e32be8780901a31679f` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:48.612880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a5b4c-4eda-431a-b8af-e9ff87dfa585 None admin] RBAC: Authorizing `identity:get_user(user_id=25374d78618f4e32be8780901a31679f)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:48.614358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a5b4c-4eda-431a-b8af-e9ff87dfa585 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:48.615898 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 8/17] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:52:48 2022] GET /identity/v3/users/25374d78618f4e32be8780901a31679f => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:52:48.621769 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a5b4c-4eda-431a-b8af-e9ff87dfa585 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:48.637902 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:48.693151 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1d297-432d-4ef0-8521-0db88a704144 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:48.721037 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a5b4c-4eda-431a-b8af-e9ff87dfa585 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:48.725941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638f409-d539-4001-90cd-3b115ab390b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:48.726822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638f409-d539-4001-90cd-3b115ab390b7 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:48.727100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638f409-d539-4001-90cd-3b115ab390b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:48.727370 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638f409-d539-4001-90cd-3b115ab390b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:48.745747 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638f409-d539-4001-90cd-3b115ab390b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:48.747285 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638f409-d539-4001-90cd-3b115ab390b7 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:48.753426 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e6538-a750-4c7a-ac19-73fa668e400a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:48.754087 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e6538-a750-4c7a-ac19-73fa668e400a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:48.754367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e6538-a750-4c7a-ac19-73fa668e400a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:48.754638 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e6538-a750-4c7a-ac19-73fa668e400a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:48.755737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e6538-a750-4c7a-ac19-73fa668e400a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:48.757175 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e6538-a750-4c7a-ac19-73fa668e400a None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:48.758775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 9/18] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:52:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:52:48.764699 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e6538-a750-4c7a-ac19-73fa668e400a None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:48.765312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-9638f409-d539-4001-90cd-3b115ab390b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:52:48.765312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:52:48.765312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:52:48.765312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:52:48.765312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:52:48.765312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:52:48.765312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:52:48.765312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:52:48.765312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:52:48.765312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:52:48.765312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:52:48.765312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:52:48.765312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:52:48.765312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:52:48.765312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:52:48.765312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:52:48.766176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:52:48.766176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:52:48.766176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:52:48.766176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:52:48.766176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:52:48.766176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:52:48.766176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:52:48.766176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:52:48.766176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:52:48.766176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:52:48.766176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:52:48.766176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:52:48.766176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:52:48.766176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:52:48.766176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:52:48.766176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:52:48.766176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:52:48.771758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:52:48.771758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:52:48.771758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:52:48.771758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 10/19] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:52:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:52:48.771984 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638f409-d539-4001-90cd-3b115ab390b7 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:48.826854 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:48.845004 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:48.864902 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e6538-a750-4c7a-ac19-73fa668e400a None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:48.871702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638f409-d539-4001-90cd-3b115ab390b7 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:48.897680 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b512be-653b-4a28-a53d-cb5b8eff9d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:48.898365 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b512be-653b-4a28-a53d-cb5b8eff9d5b None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:48.898649 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b512be-653b-4a28-a53d-cb5b8eff9d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:48.898919 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b512be-653b-4a28-a53d-cb5b8eff9d5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:48.900152 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b512be-653b-4a28-a53d-cb5b8eff9d5b None admin] RBAC: Adding query filter params (role.id=aa420c2fb1ab4bb3b05b004a4d89779a, scope.domain.id=default, user.id=25374d78618f4e32be8780901a31679f) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:52:48.900455 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b512be-653b-4a28-a53d-cb5b8eff9d5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:48.901887 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b512be-653b-4a28-a53d-cb5b8eff9d5b None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:48.904404 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6b05f-20d9-44e0-ae0f-dd0dc6ddc915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:48.905009 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6b05f-20d9-44e0-ae0f-dd0dc6ddc915 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:48.905287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6b05f-20d9-44e0-ae0f-dd0dc6ddc915 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:48.905568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6b05f-20d9-44e0-ae0f-dd0dc6ddc915 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:48.906237 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6b05f-20d9-44e0-ae0f-dd0dc6ddc915 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:52:48.906539 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6b05f-20d9-44e0-ae0f-dd0dc6ddc915 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:48.907917 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6b05f-20d9-44e0-ae0f-dd0dc6ddc915 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:48.908678 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 10/20] 10.222.0.15 () {56 vars in 1322 bytes} [Wed Sep 14 01:52:48 2022] GET /identity/v3/role_assignments?user.id=25374d78618f4e32be8780901a31679f&scope.domain.id=default&role.id=aa420c2fb1ab4bb3b05b004a4d89779a => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:52:48.927259 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 11/21] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:52:51.518741 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28721d34-8998-45a1-915b-f8ce74d77e41 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:51.519239 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28721d34-8998-45a1-915b-f8ce74d77e41 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:51.519311 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28721d34-8998-45a1-915b-f8ce74d77e41 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:51.520640 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 11/22] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:52:51.524352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc840ae-3269-452f-8f01-bac1a719e475 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:51.525219 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc840ae-3269-452f-8f01-bac1a719e475 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:51.525698 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc840ae-3269-452f-8f01-bac1a719e475 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:51.527244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 12/23] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:52:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:52:51.530821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188dffd-1910-4538-90e9-f6bea26ad5ec None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:51.531106 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188dffd-1910-4538-90e9-f6bea26ad5ec None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:51.531380 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188dffd-1910-4538-90e9-f6bea26ad5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:51.552715 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-8188dffd-1910-4538-90e9-f6bea26ad5ec None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:52:51.553907 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-8188dffd-1910-4538-90e9-f6bea26ad5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:52:51.555900 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb6654-4a62-4a5e-b43c-d6c4e81d7974 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:51.556411 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb6654-4a62-4a5e-b43c-d6c4e81d7974 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:51.556900 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb6654-4a62-4a5e-b43c-d6c4e81d7974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:51.590977 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-4dcb6654-4a62-4a5e-b43c-d6c4e81d7974 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:52:51.592511 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-4dcb6654-4a62-4a5e-b43c-d6c4e81d7974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:52:51.628486 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 12/24] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:52:51.732331 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 13/25] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:52:51.974584 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188dffd-1910-4538-90e9-f6bea26ad5ec None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:52.051332 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:52.077517 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188dffd-1910-4538-90e9-f6bea26ad5ec None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:52.115698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4c5b2-11b7-4256-a480-084cb705e0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:52.116727 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4c5b2-11b7-4256-a480-084cb705e0cf None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:52.117224 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4c5b2-11b7-4256-a480-084cb705e0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:52.117770 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4c5b2-11b7-4256-a480-084cb705e0cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:52.129692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4c5b2-11b7-4256-a480-084cb705e0cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:52:52.135880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-09b4c5b2-11b7-4256-a480-084cb705e0cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:52:52.135880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:52:52.135880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:52:52.135880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:52:52.135880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:52:52.135880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:52:52.135880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:52:52.135880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:52:52.135880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:52:52.135880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:52:52.135880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:52:52.135880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:52:52.135880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:52:52.135880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:52:52.135880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:52:52.135880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:52:52.136748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:52:52.136748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:52:52.136748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:52:52.136748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:52:52.136748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:52:52.136748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:52:52.136748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:52:52.136748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:52:52.136748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:52:52.136748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:52:52.136748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:52:52.136748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:52:52.136748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:52:52.136748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:52:52.136748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:52:52.136748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:52:52.137529 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:52:52.137529 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:52:52.137529 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:52:52.137529 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:52:52.137529 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:52:52.137529 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:52:52.137529 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:52:52.137529 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:52:52.137529 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:52:52.137529 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:52:52.137529 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:52:52.137529 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:52:52.139821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 13/26] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:52:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:52:52.145455 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e924a69-9f0a-40d2-ab75-f37a216b4ab6 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:52.145967 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e924a69-9f0a-40d2-ab75-f37a216b4ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:52.146493 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e924a69-9f0a-40d2-ab75-f37a216b4ab6 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:52.148038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 14/27] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:52:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:52:52.149167 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4c5b2-11b7-4256-a480-084cb705e0cf None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:52.154605 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481e79e-01e6-430b-b6cc-4c1ad3baaf4a None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:52.155180 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481e79e-01e6-430b-b6cc-4c1ad3baaf4a None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:52.155649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481e79e-01e6-430b-b6cc-4c1ad3baaf4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:52.183452 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-e481e79e-01e6-430b-b6cc-4c1ad3baaf4a None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:52:52.186814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-e481e79e-01e6-430b-b6cc-4c1ad3baaf4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:52:52.263544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:52.292690 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 15/28] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:52:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:52:52.293559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481e79e-01e6-430b-b6cc-4c1ad3baaf4a None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:52.307871 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4c5b2-11b7-4256-a480-084cb705e0cf None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:52.343026 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70706712-7881-4dda-a3e8-f1cb3ab6137c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:52.343978 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70706712-7881-4dda-a3e8-f1cb3ab6137c None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:52.344263 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70706712-7881-4dda-a3e8-f1cb3ab6137c None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:52.344539 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70706712-7881-4dda-a3e8-f1cb3ab6137c None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:52.345365 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70706712-7881-4dda-a3e8-f1cb3ab6137c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:52:52.345945 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70706712-7881-4dda-a3e8-f1cb3ab6137c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:52.347400 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70706712-7881-4dda-a3e8-f1cb3ab6137c None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:52.356151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 14/29] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:52:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:52:52.357631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7493476-6a8e-47f3-b549-c9c147e6e28c None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:52.357927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7493476-6a8e-47f3-b549-c9c147e6e28c None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:52.358229 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7493476-6a8e-47f3-b549-c9c147e6e28c None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:52.359581 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 15/30] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:52:52.360070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:52.364561 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7493476-6a8e-47f3-b549-c9c147e6e28c None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:52.428991 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481e79e-01e6-430b-b6cc-4c1ad3baaf4a None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:52.465332 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819dde68-4608-4c2c-862f-43fe2d4ec185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:52.466230 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819dde68-4608-4c2c-862f-43fe2d4ec185 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:52.466660 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819dde68-4608-4c2c-862f-43fe2d4ec185 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:52.467075 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819dde68-4608-4c2c-862f-43fe2d4ec185 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:52.477930 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819dde68-4608-4c2c-862f-43fe2d4ec185 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:52.479744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819dde68-4608-4c2c-862f-43fe2d4ec185 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:52.511621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-819dde68-4608-4c2c-862f-43fe2d4ec185 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:52:52.511621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:52:52.511621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:52:52.511621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:52:52.511621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:52:52.511621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:52:52.511621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:52:52.511621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:52:52.511621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:52:52.511621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:52:52.511621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:52:52.511621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:52:52.511621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:52:52.511621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:52:52.511621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:52:52.511621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:52:52.512736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:52:52.512736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:52:52.512736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:52:52.512736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:52:52.512736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:52:52.512736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:52:52.512736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:52:52.512736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:52:52.512736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:52:52.512736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:52:52.512736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:52:52.512736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:52:52.512736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:52:52.512736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:52:52.512736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:52:52.512736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:52:52.512736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:52:52.513762 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:52:52.513762 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:52:52.513762 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:52:52.515176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 16/31] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:52:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:52:52.522027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1edf7-f4b7-4d7f-9f5a-34e4d0c1ab9b None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:52.522627 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1edf7-f4b7-4d7f-9f5a-34e4d0c1ab9b None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:52.523786 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1edf7-f4b7-4d7f-9f5a-34e4d0c1ab9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:52.544370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7493476-6a8e-47f3-b549-c9c147e6e28c None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:52.561555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-1af1edf7-f4b7-4d7f-9f5a-34e4d0c1ab9b None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:52:52.564542 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-1af1edf7-f4b7-4d7f-9f5a-34e4d0c1ab9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:52:52.606918 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391ea4f-3665-4789-99da-bd4e6b33a730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:52.608948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391ea4f-3665-4789-99da-bd4e6b33a730 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:52.610088 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391ea4f-3665-4789-99da-bd4e6b33a730 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:52.610913 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391ea4f-3665-4789-99da-bd4e6b33a730 None admin] PATH_INFO: `/v3/users/25374d78618f4e32be8780901a31679f` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:52.613604 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391ea4f-3665-4789-99da-bd4e6b33a730 None admin] RBAC: Authorizing `identity:get_user(user_id=25374d78618f4e32be8780901a31679f)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:52.615659 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391ea4f-3665-4789-99da-bd4e6b33a730 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:52.618370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 16/32] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:52:52 2022] GET /identity/v3/users/25374d78618f4e32be8780901a31679f => generated 247 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:52:52.620461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391ea4f-3665-4789-99da-bd4e6b33a730 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:52.631088 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:52.658695 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 17/33] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:52:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:52:52.659774 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1edf7-f4b7-4d7f-9f5a-34e4d0c1ab9b None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:52.719446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:52.794160 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391ea4f-3665-4789-99da-bd4e6b33a730 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:52.816380 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1edf7-f4b7-4d7f-9f5a-34e4d0c1ab9b None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:52.855894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177b6dd-fd19-4050-92d5-12f1f1d37daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:52.856889 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177b6dd-fd19-4050-92d5-12f1f1d37daa None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:52.857366 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177b6dd-fd19-4050-92d5-12f1f1d37daa None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:52.857834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177b6dd-fd19-4050-92d5-12f1f1d37daa None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:52.858775 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f177b6dd-fd19-4050-92d5-12f1f1d37daa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:52.860441 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f177b6dd-fd19-4050-92d5-12f1f1d37daa None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:52.869646 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38cfd2-90e4-4452-b11a-374a6766f89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:52.870928 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38cfd2-90e4-4452-b11a-374a6766f89c None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:52.871462 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38cfd2-90e4-4452-b11a-374a6766f89c None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:52.871973 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38cfd2-90e4-4452-b11a-374a6766f89c None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:52.872994 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38cfd2-90e4-4452-b11a-374a6766f89c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:52:52.873523 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38cfd2-90e4-4452-b11a-374a6766f89c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:52.881520 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38cfd2-90e4-4452-b11a-374a6766f89c None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:52.930354 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 17/34] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:52:52.931290 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38cfd2-90e4-4452-b11a-374a6766f89c None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:52.999226 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 18/35] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Sep 14 01:52:52 2022] POST /identity/v3/roles => generated 225 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:52:53.000249 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177b6dd-fd19-4050-92d5-12f1f1d37daa None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:53.022772 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:53.102622 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38cfd2-90e4-4452-b11a-374a6766f89c None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:53.141007 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:53.147532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177b6dd-fd19-4050-92d5-12f1f1d37daa None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:53.159610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992998a-ea53-46ea-91c8-f1086d714523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:53.161086 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992998a-ea53-46ea-91c8-f1086d714523 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:53.161809 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992998a-ea53-46ea-91c8-f1086d714523 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:53.162313 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992998a-ea53-46ea-91c8-f1086d714523 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:53.163955 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992998a-ea53-46ea-91c8-f1086d714523 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:53.166473 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992998a-ea53-46ea-91c8-f1086d714523 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:53.169101 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 18/36] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:52:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:52:53.170277 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992998a-ea53-46ea-91c8-f1086d714523 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:53.190366 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33956f1-e650-4169-8b72-ffe813cfe46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:53.191145 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33956f1-e650-4169-8b72-ffe813cfe46a None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:53.191515 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33956f1-e650-4169-8b72-ffe813cfe46a None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:53.191845 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33956f1-e650-4169-8b72-ffe813cfe46a None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:53.192908 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33956f1-e650-4169-8b72-ffe813cfe46a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:53.194622 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33956f1-e650-4169-8b72-ffe813cfe46a None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:53.231591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 19/37] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 14 01:52:52 2022] POST /identity/v3/projects => generated 301 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:52:53.232242 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def41880-5169-48bd-9e28-5498a9f1ebf1 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:53.232568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def41880-5169-48bd-9e28-5498a9f1ebf1 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:53.232847 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def41880-5169-48bd-9e28-5498a9f1ebf1 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:53.234100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 20/38] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:52:53.238177 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def41880-5169-48bd-9e28-5498a9f1ebf1 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:53.314847 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992998a-ea53-46ea-91c8-f1086d714523 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:53.354281 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:53.364570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc093063-084e-42d4-83b1-da9ab25207c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:53.365206 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc093063-084e-42d4-83b1-da9ab25207c0 None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:53.365484 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc093063-084e-42d4-83b1-da9ab25207c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:53.365756 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc093063-084e-42d4-83b1-da9ab25207c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:53.366479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc093063-084e-42d4-83b1-da9ab25207c0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:53.369627 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc093063-084e-42d4-83b1-da9ab25207c0 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:53.406564 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def41880-5169-48bd-9e28-5498a9f1ebf1 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:53.454448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d5b0a-f198-4179-95a0-666cb982a04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:53.455115 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d5b0a-f198-4179-95a0-666cb982a04b None admin] REQUEST_METHOD: `PUT` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:53.455402 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d5b0a-f198-4179-95a0-666cb982a04b None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:53.455682 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d5b0a-f198-4179-95a0-666cb982a04b None admin] PATH_INFO: `/v3/domains/default/users/25374d78618f4e32be8780901a31679f/roles/aa420c2fb1ab4bb3b05b004a4d89779a` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:53.457855 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d5b0a-f198-4179-95a0-666cb982a04b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=25374d78618f4e32be8780901a31679f, role_id=aa420c2fb1ab4bb3b05b004a4d89779a)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:53.459430 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d5b0a-f198-4179-95a0-666cb982a04b None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:53.480383 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 21/39] 10.222.0.15 () {58 vars in 1361 bytes} [Wed Sep 14 01:52:53 2022] PUT /identity/v3/domains/default/users/25374d78618f4e32be8780901a31679f/roles/aa420c2fb1ab4bb3b05b004a4d89779a => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:52:53.482036 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882c77a-9a62-4651-af53-0df8160217e1 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:53.482517 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882c77a-9a62-4651-af53-0df8160217e1 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:53.482972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882c77a-9a62-4651-af53-0df8160217e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:53.504409 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 19/40] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Sep 14 01:52:53 2022] POST /identity/v3/roles => generated 219 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:52:53.506052 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663dd82-69b1-49e4-b5f1-bb29d7ee61cf None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:53.506354 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663dd82-69b1-49e4-b5f1-bb29d7ee61cf None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:53.506633 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663dd82-69b1-49e4-b5f1-bb29d7ee61cf None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:53.508229 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 20/41] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:52:53.508492 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-e882c77a-9a62-4651-af53-0df8160217e1 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:52:53.510872 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-e882c77a-9a62-4651-af53-0df8160217e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:52:53.514335 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c920e08-404a-49de-9bba-d0377e700474 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:53.514658 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c920e08-404a-49de-9bba-d0377e700474 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:53.514895 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c920e08-404a-49de-9bba-d0377e700474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:53.607688 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 22/42] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:52:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:52:53.649637 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-4c920e08-404a-49de-9bba-d0377e700474 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:52:53.651303 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-4c920e08-404a-49de-9bba-d0377e700474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:52:53.709384 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:53.735188 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 21/43] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:52:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:52:53.892267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882c77a-9a62-4651-af53-0df8160217e1 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:53.952127 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:53.994577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882c77a-9a62-4651-af53-0df8160217e1 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:54.014417 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c920e08-404a-49de-9bba-d0377e700474 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:54.033721 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263b6d6-59f4-4fb4-be93-b79e3c713bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:54.034910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263b6d6-59f4-4fb4-be93-b79e3c713bb0 None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:54.035401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263b6d6-59f4-4fb4-be93-b79e3c713bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:54.035853 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263b6d6-59f4-4fb4-be93-b79e3c713bb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:54.036705 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263b6d6-59f4-4fb4-be93-b79e3c713bb0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:54.039024 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263b6d6-59f4-4fb4-be93-b79e3c713bb0 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:54.057008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 23/44] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Sep 14 01:52:53 2022] POST /identity/v3/roles => generated 223 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:52:54.125968 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c920e08-404a-49de-9bba-d0377e700474 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:54.162512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0d8e1-1244-4cbc-acaf-cc52624d816d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:54.163081 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0d8e1-1244-4cbc-acaf-cc52624d816d None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:54.163361 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0d8e1-1244-4cbc-acaf-cc52624d816d None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:54.163631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0d8e1-1244-4cbc-acaf-cc52624d816d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:54.164811 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0d8e1-1244-4cbc-acaf-cc52624d816d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:54.166466 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0d8e1-1244-4cbc-acaf-cc52624d816d None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:54.168273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 22/45] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:52:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:52:54.173930 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263b6d6-59f4-4fb4-be93-b79e3c713bb0 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:54.285614 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263b6d6-59f4-4fb4-be93-b79e3c713bb0 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:54.306681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:54.321497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714fef95-3e78-4e5d-a876-bd571f322243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:54.322148 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fef95-3e78-4e5d-a876-bd571f322243 None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:54.322433 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fef95-3e78-4e5d-a876-bd571f322243 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:54.322708 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fef95-3e78-4e5d-a876-bd571f322243 None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:54.323364 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714fef95-3e78-4e5d-a876-bd571f322243 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:54.324825 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714fef95-3e78-4e5d-a876-bd571f322243 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:54.350575 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 24/46] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 14 01:52:54 2022] POST /identity/v3/projects => generated 312 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:52:55.681945 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca459769-7556-4336-9d5a-195714830d64 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:55.682476 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca459769-7556-4336-9d5a-195714830d64 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:55.682476 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca459769-7556-4336-9d5a-195714830d64 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:55.683769 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 23/47] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:52:55.688892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f687679-fb41-4d71-8d41-2edf9471411b None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:55.689301 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f687679-fb41-4d71-8d41-2edf9471411b None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:55.689449 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f687679-fb41-4d71-8d41-2edf9471411b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:55.707311 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-6f687679-fb41-4d71-8d41-2edf9471411b None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:52:55.708485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-6f687679-fb41-4d71-8d41-2edf9471411b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:52:55.772752 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 25/48] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:52:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:52:56.041331 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca459769-7556-4336-9d5a-195714830d64 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:56.091337 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:56.142143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca459769-7556-4336-9d5a-195714830d64 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:56.174287 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b624d-3d32-4215-84fa-0fb6b7276c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:56.174911 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b624d-3d32-4215-84fa-0fb6b7276c7a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:56.175191 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b624d-3d32-4215-84fa-0fb6b7276c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:56.175463 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b624d-3d32-4215-84fa-0fb6b7276c7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:56.185752 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b624d-3d32-4215-84fa-0fb6b7276c7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:52:56.190556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-a07b624d-3d32-4215-84fa-0fb6b7276c7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:52:56.190556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:52:56.190556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:52:56.190556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:52:56.190556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:52:56.190556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:52:56.190556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:52:56.190556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:52:56.190556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:52:56.190556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:52:56.190556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:52:56.190556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:52:56.190556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:52:56.190556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:52:56.190556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:52:56.190556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:52:56.191562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:52:56.191562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:52:56.191562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:52:56.191562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:52:56.191562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:52:56.191562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:52:56.191562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:52:56.191562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:52:56.191562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:52:56.191562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:52:56.191562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:52:56.191562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:52:56.191562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:52:56.191562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:52:56.191562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:52:56.191562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:52:56.192358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:52:56.192358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:52:56.192358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:52:56.192358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:52:56.192358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:52:56.192358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:52:56.192358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:52:56.192358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:52:56.192358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:52:56.192358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:52:56.192358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:52:56.192358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:52:56.192358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 24/49] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:52:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:52:56.197943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f687679-fb41-4d71-8d41-2edf9471411b None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:56.293506 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f687679-fb41-4d71-8d41-2edf9471411b None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:56.294955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:56.329506 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2dec5-efa2-4d05-a79f-71efa7e463ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:56.330153 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2dec5-efa2-4d05-a79f-71efa7e463ac None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:56.330436 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2dec5-efa2-4d05-a79f-71efa7e463ac None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:56.330711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2dec5-efa2-4d05-a79f-71efa7e463ac None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:56.331369 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2dec5-efa2-4d05-a79f-71efa7e463ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:52:56.331674 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2dec5-efa2-4d05-a79f-71efa7e463ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:56.333066 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2dec5-efa2-4d05-a79f-71efa7e463ac None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:56.344458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 26/50] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:52:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:52:56.350787 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b624d-3d32-4215-84fa-0fb6b7276c7a None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:56.456512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b624d-3d32-4215-84fa-0fb6b7276c7a None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:56.466894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:56.491633 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c19928-9d40-4bce-8e4f-b0d03f6384e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:56.492866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c19928-9d40-4bce-8e4f-b0d03f6384e2 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:56.493158 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c19928-9d40-4bce-8e4f-b0d03f6384e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:56.493433 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c19928-9d40-4bce-8e4f-b0d03f6384e2 None admin] PATH_INFO: `/v3/users/25374d78618f4e32be8780901a31679f` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:56.494591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c19928-9d40-4bce-8e4f-b0d03f6384e2 None admin] RBAC: Authorizing `identity:get_user(user_id=25374d78618f4e32be8780901a31679f)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:56.496065 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c19928-9d40-4bce-8e4f-b0d03f6384e2 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:56.496262 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449e99b-d664-4168-8cb5-7bff3aff88a0 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:56.496553 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449e99b-d664-4168-8cb5-7bff3aff88a0 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:56.496826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449e99b-d664-4168-8cb5-7bff3aff88a0 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:56.497632 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 25/51] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:52:56 2022] GET /identity/v3/users/25374d78618f4e32be8780901a31679f => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:52:56.498204 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 27/52] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:52:56.502778 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c19928-9d40-4bce-8e4f-b0d03f6384e2 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:56.503211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33a764-f0b8-4c77-b13f-f1d4596ee9b8 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:56.503499 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33a764-f0b8-4c77-b13f-f1d4596ee9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:56.503772 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33a764-f0b8-4c77-b13f-f1d4596ee9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:56.522352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-6d33a764-f0b8-4c77-b13f-f1d4596ee9b8 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:52:56.523552 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-6d33a764-f0b8-4c77-b13f-f1d4596ee9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:52:56.607552 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c19928-9d40-4bce-8e4f-b0d03f6384e2 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:56.646531 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ad57d-d784-42cf-bace-c45ccf58644d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:56.647186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ad57d-d784-42cf-bace-c45ccf58644d None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:56.647470 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ad57d-d784-42cf-bace-c45ccf58644d None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:56.647742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ad57d-d784-42cf-bace-c45ccf58644d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:56.648966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ad57d-d784-42cf-bace-c45ccf58644d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:56.650491 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ad57d-d784-42cf-bace-c45ccf58644d None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:56.652095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 26/53] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:52:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:52:56.657888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ad57d-d784-42cf-bace-c45ccf58644d None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:56.692002 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 28/54] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:52:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:52:56.704687 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:56.756807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ad57d-d784-42cf-bace-c45ccf58644d None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:56.789704 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ec339-1661-4112-a9ca-6962ab1a6f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:56.790340 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ec339-1661-4112-a9ca-6962ab1a6f71 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:56.790623 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ec339-1661-4112-a9ca-6962ab1a6f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:56.790897 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ec339-1661-4112-a9ca-6962ab1a6f71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:56.791609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ec339-1661-4112-a9ca-6962ab1a6f71 None admin] RBAC: Adding query filter params (role.id=aa420c2fb1ab4bb3b05b004a4d89779a, scope.domain.id=default, user.id=25374d78618f4e32be8780901a31679f) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:52:56.791915 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ec339-1661-4112-a9ca-6962ab1a6f71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:56.793361 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ec339-1661-4112-a9ca-6962ab1a6f71 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:56.798957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 27/55] 10.222.0.15 () {56 vars in 1322 bytes} [Wed Sep 14 01:52:56 2022] GET /identity/v3/role_assignments?user.id=25374d78618f4e32be8780901a31679f&scope.domain.id=default&role.id=aa420c2fb1ab4bb3b05b004a4d89779a => generated 530 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:52:56.963281 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33a764-f0b8-4c77-b13f-f1d4596ee9b8 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:57.055903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:57.063040 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33a764-f0b8-4c77-b13f-f1d4596ee9b8 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:57.095428 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5e3d5-9cf8-42fa-b60d-29686ae0a59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:57.096059 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5e3d5-9cf8-42fa-b60d-29686ae0a59f None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:57.096340 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5e3d5-9cf8-42fa-b60d-29686ae0a59f None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:57.096616 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5e3d5-9cf8-42fa-b60d-29686ae0a59f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:57.097711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5e3d5-9cf8-42fa-b60d-29686ae0a59f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:57.099193 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5e3d5-9cf8-42fa-b60d-29686ae0a59f None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:57.100757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 29/56] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:52:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:52:57.106453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ec339-1661-4112-a9ca-6962ab1a6f71 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:57.203220 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ec339-1661-4112-a9ca-6962ab1a6f71 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:57.213090 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:57.236189 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a6c05-c91c-4d14-ba6b-8b2dac87e541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:57.236732 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a6c05-c91c-4d14-ba6b-8b2dac87e541 None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:57.237014 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a6c05-c91c-4d14-ba6b-8b2dac87e541 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:57.237287 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a6c05-c91c-4d14-ba6b-8b2dac87e541 None admin] PATH_INFO: `/v3/projects` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:57.237936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a6c05-c91c-4d14-ba6b-8b2dac87e541 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:57.239421 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a6c05-c91c-4d14-ba6b-8b2dac87e541 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:57.266591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 28/57] 10.222.0.15 () {60 vars in 1140 bytes} [Wed Sep 14 01:52:57 2022] POST /identity/v3/projects => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:52:59.403704 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba923725-940c-4f4d-8e1a-7be4ae26e24a None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:59.404187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba923725-940c-4f4d-8e1a-7be4ae26e24a None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:59.404277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba923725-940c-4f4d-8e1a-7be4ae26e24a None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:59.405598 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 30/58] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:52:59.410931 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ce553-5643-4c0a-82bb-47ac747d8b03 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:59.411332 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ce553-5643-4c0a-82bb-47ac747d8b03 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:59.411494 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ce553-5643-4c0a-82bb-47ac747d8b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:59.429347 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-d13ce553-5643-4c0a-82bb-47ac747d8b03 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:52:59.431097 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-d13ce553-5643-4c0a-82bb-47ac747d8b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:52:59.495522 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 29/59] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:52:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:52:59.759291 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba923725-940c-4f4d-8e1a-7be4ae26e24a None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:59.856455 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba923725-940c-4f4d-8e1a-7be4ae26e24a None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:52:59.888462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c62c08-70fc-44d6-8352-a53acf910e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:52:59.889088 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c62c08-70fc-44d6-8352-a53acf910e3e None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:52:59.889368 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c62c08-70fc-44d6-8352-a53acf910e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:52:59.889636 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c62c08-70fc-44d6-8352-a53acf910e3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:52:59.890756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c62c08-70fc-44d6-8352-a53acf910e3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:52:59.892190 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c62c08-70fc-44d6-8352-a53acf910e3e None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:52:59.893717 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 31/60] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:52:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:52:59.899552 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ce553-5643-4c0a-82bb-47ac747d8b03 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:52:59.986522 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:52:59.996245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ce553-5643-4c0a-82bb-47ac747d8b03 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:00.029154 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35ec7a-4fb9-4f87-bcf8-a6ee9111eb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:00.029703 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35ec7a-4fb9-4f87-bcf8-a6ee9111eb61 None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:00.030012 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35ec7a-4fb9-4f87-bcf8-a6ee9111eb61 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:00.030287 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35ec7a-4fb9-4f87-bcf8-a6ee9111eb61 None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:00.030929 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e35ec7a-4fb9-4f87-bcf8-a6ee9111eb61 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:00.032390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e35ec7a-4fb9-4f87-bcf8-a6ee9111eb61 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:00.058203 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 30/61] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 14 01:52:59 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:53:03.191795 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654078fa-ba09-4858-8ecd-2d461bdf42cc None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:03.192840 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654078fa-ba09-4858-8ecd-2d461bdf42cc None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:03.193326 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654078fa-ba09-4858-8ecd-2d461bdf42cc None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:03.195459 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 32/62] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:03.204373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b1d74-6940-4cdb-a15b-dc9986b3da48 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:03.204828 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b1d74-6940-4cdb-a15b-dc9986b3da48 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:03.205016 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b1d74-6940-4cdb-a15b-dc9986b3da48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:03.236443 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-5b1b1d74-6940-4cdb-a15b-dc9986b3da48 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:03.237667 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-5b1b1d74-6940-4cdb-a15b-dc9986b3da48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:03.328269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 31/63] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:03.673211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2264c2-1368-4969-b9f6-f97973301ee9 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:03.673909 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2264c2-1368-4969-b9f6-f97973301ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:03.674446 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2264c2-1368-4969-b9f6-f97973301ee9 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:03.676137 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 33/64] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:03.682897 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f3503-717a-492d-9d54-4734e541dd1b None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:03.683182 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f3503-717a-492d-9d54-4734e541dd1b None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:03.683626 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f3503-717a-492d-9d54-4734e541dd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:03.689595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580ec58-d4a7-4704-a9d5-605d15721647 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:03.689924 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580ec58-d4a7-4704-a9d5-605d15721647 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:03.690197 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580ec58-d4a7-4704-a9d5-605d15721647 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:03.691572 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 34/65] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:53:03.706486 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07604cb-8332-48b0-90e6-90e3a587860f None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:03.706778 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07604cb-8332-48b0-90e6-90e3a587860f None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:03.707041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07604cb-8332-48b0-90e6-90e3a587860f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:03.711775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-3d1f3503-717a-492d-9d54-4734e541dd1b None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:03.712981 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-3d1f3503-717a-492d-9d54-4734e541dd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:03.737905 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-c07604cb-8332-48b0-90e6-90e3a587860f None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:03.739329 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-c07604cb-8332-48b0-90e6-90e3a587860f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:03.766359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:03.826350 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 32/66] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:03.827051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f3503-717a-492d-9d54-4734e541dd1b None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:03.834538 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 35/67] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:03.836357 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a478d-35da-4503-91d4-fc6e2ef3539d None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:03.836648 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a478d-35da-4503-91d4-fc6e2ef3539d None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:03.836914 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a478d-35da-4503-91d4-fc6e2ef3539d None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:03.838275 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 36/68] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:03.843159 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:03.848815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed551176-3669-414a-84dd-305a0acee0ad None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:03.849377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed551176-3669-414a-84dd-305a0acee0ad None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:03.851322 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed551176-3669-414a-84dd-305a0acee0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:03.899371 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-ed551176-3669-414a-84dd-305a0acee0ad None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:03.900580 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-ed551176-3669-414a-84dd-305a0acee0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:03.938517 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f3503-717a-492d-9d54-4734e541dd1b None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:03.973788 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd44c6-77cc-4199-a5a7-ef7cfdd843bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:03.974723 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd44c6-77cc-4199-a5a7-ef7cfdd843bf None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:03.975150 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd44c6-77cc-4199-a5a7-ef7cfdd843bf None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:03.975567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd44c6-77cc-4199-a5a7-ef7cfdd843bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:03.987240 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd44c6-77cc-4199-a5a7-ef7cfdd843bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:03.992896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-b2cd44c6-77cc-4199-a5a7-ef7cfdd843bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:03.992896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:03.992896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:03.992896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:03.992896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:03.992896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:03.992896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:03.992896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:03.992896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:03.992896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:03.992896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:03.992896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:03.992896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:03.992896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:03.992896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:03.992896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:03.993936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:03.993936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:03.993936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:03.993936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:03.993936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:03.993936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:03.993936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:03.993936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:03.993936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:03.993936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:03.993936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:03.993936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:03.993936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:03.993936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:03.993936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:03.993936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:03.994930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:03.994930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:03.994930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:03.994930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:03.994930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:03.994930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:03.994930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:03.994930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:03.994930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:03.994930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:03.994930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:03.994930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:03.996786 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 33/69] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:03.998607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e69bf-31ea-471d-bf64-80772405587d None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:03.999037 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e69bf-31ea-471d-bf64-80772405587d None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:03.999471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e69bf-31ea-471d-bf64-80772405587d None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:04.000926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 34/70] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:04.002327 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e69bf-31ea-471d-bf64-80772405587d None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:04.015842 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 37/71] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:04.023411 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0448b38-6bba-482e-971e-089294933b02 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:04.023701 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0448b38-6bba-482e-971e-089294933b02 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:04.024192 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0448b38-6bba-482e-971e-089294933b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:04.048769 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-c0448b38-6bba-482e-971e-089294933b02 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:04.050291 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-c0448b38-6bba-482e-971e-089294933b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:04.088682 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:04.124307 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 38/72] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:04.124636 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e69bf-31ea-471d-bf64-80772405587d None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:04.160069 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4f13b-50aa-4577-b844-5448871f31a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:04.160931 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4f13b-50aa-4577-b844-5448871f31a4 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:04.161358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4f13b-50aa-4577-b844-5448871f31a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:04.161776 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4f13b-50aa-4577-b844-5448871f31a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:04.162601 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4f13b-50aa-4577-b844-5448871f31a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:04.163103 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4f13b-50aa-4577-b844-5448871f31a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:04.165266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4f13b-50aa-4577-b844-5448871f31a4 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:04.165397 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0448b38-6bba-482e-971e-089294933b02 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:04.174583 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 35/73] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:04.182161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4f13b-50aa-4577-b844-5448871f31a4 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:04.200202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:04.278524 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0448b38-6bba-482e-971e-089294933b02 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:04.289657 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4f13b-50aa-4577-b844-5448871f31a4 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:04.320848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:04.323894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f41f5-68e5-40ac-a970-f3a73f311ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:04.324751 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f41f5-68e5-40ac-a970-f3a73f311ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:04.325039 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f41f5-68e5-40ac-a970-f3a73f311ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:04.325313 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f41f5-68e5-40ac-a970-f3a73f311ba5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:04.327816 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e28417-8841-49b4-ae82-9139090ca1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:04.328598 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e28417-8841-49b4-ae82-9139090ca1c7 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:04.329273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e28417-8841-49b4-ae82-9139090ca1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:04.329688 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e28417-8841-49b4-ae82-9139090ca1c7 None admin] PATH_INFO: `/v3/users/77fa2fe74be84241a0d5150ae86101fb` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:04.336528 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f41f5-68e5-40ac-a970-f3a73f311ba5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:04.342911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-694f41f5-68e5-40ac-a970-f3a73f311ba5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:04.342911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:04.342911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:04.342911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:04.342911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:04.342911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:04.342911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:04.342911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:04.342911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:04.342911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:04.342911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:04.342911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:04.342911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:04.342911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:04.342911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:04.342911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:04.343780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:04.343780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:04.343780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:04.343780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:04.343780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:04.343780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:04.343780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:04.343780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:04.343780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:04.343780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:04.343780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:04.343780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:04.343780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:04.343780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:04.343780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:04.343780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:04.344607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:04.344607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:04.344607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:04.344607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:04.344607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:04.344607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:04.344607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:04.344607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:04.344607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:04.344607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:04.344607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:04.344607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:04.344607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 39/74] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:53:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:04.347255 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e28417-8841-49b4-ae82-9139090ca1c7 None admin] RBAC: Authorizing `identity:get_user(user_id=77fa2fe74be84241a0d5150ae86101fb)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:04.348883 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e28417-8841-49b4-ae82-9139090ca1c7 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:04.349427 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f41f5-68e5-40ac-a970-f3a73f311ba5 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:04.354030 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 36/75] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:04 2022] GET /identity/v3/users/77fa2fe74be84241a0d5150ae86101fb => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:04.354030 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e28417-8841-49b4-ae82-9139090ca1c7 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:04.441114 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:04.459218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e28417-8841-49b4-ae82-9139090ca1c7 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:04.459585 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f41f5-68e5-40ac-a970-f3a73f311ba5 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:04.491055 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6401a8-8cb4-432c-b6f4-af249bff6cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:04.491368 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1bde3-c797-4794-b836-fef05f51f3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:04.492221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6401a8-8cb4-432c-b6f4-af249bff6cce None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:04.492500 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6401a8-8cb4-432c-b6f4-af249bff6cce None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:04.492589 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1bde3-c797-4794-b836-fef05f51f3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:04.492783 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6401a8-8cb4-432c-b6f4-af249bff6cce None admin] PATH_INFO: `/v3/roles/member` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:04.492909 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1bde3-c797-4794-b836-fef05f51f3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:04.493173 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1bde3-c797-4794-b836-fef05f51f3f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:04.502503 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6401a8-8cb4-432c-b6f4-af249bff6cce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:04.503330 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1bde3-c797-4794-b836-fef05f51f3f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:04.507375 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-5a6401a8-8cb4-432c-b6f4-af249bff6cce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:04.507375 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:04.507375 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:04.507375 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:04.507375 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:04.507375 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:04.507375 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:04.507375 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:04.507375 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:04.507375 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:04.507375 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:04.507375 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:04.507375 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:04.507375 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:04.507375 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:04.507375 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:04.509204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:04.509204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:04.509204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:04.509204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:04.509204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:04.509204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:04.509204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:04.509204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:04.509204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:04.509204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:04.509204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:04.509204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:04.509204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:04.509204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:04.509204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:04.509204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:04.510573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:04.510573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:04.510573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:04.510573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:04.510573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:04.510573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:04.510573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:04.510573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:04.510573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:04.510573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:04.510573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:04.510573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:04.511169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-24a1bde3-c797-4794-b836-fef05f51f3f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:53:04.511169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:04.511169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:04.511169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:04.511169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:04.511169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:04.511169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:04.511169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:04.511169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:04.511169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:04.511169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:04.511169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:04.511169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:04.511169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:04.511169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:04.511169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:04.511941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:04.511941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:04.511941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:04.511941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:04.511941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:04.511941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:04.511941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:04.511941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:04.511941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:04.511941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:04.511941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:04.511941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:04.511941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:04.511941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:04.511941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:04.511941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:04.512754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:04.512754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:04.512754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:04.512754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:04.512754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:04.512754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:04.512754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:04.512754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:04.512754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:04.512754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:04.512754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:53:04.512754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:04.513424 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 37/76] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:04.513424 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6401a8-8cb4-432c-b6f4-af249bff6cce None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:04.513550 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 40/77] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:53:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:04.513550 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1bde3-c797-4794-b836-fef05f51f3f2 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:04.564431 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:04.611171 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1bde3-c797-4794-b836-fef05f51f3f2 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:04.613626 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6401a8-8cb4-432c-b6f4-af249bff6cce None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:04.645519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5e985-d20a-43b5-9c65-e0b5e62e3016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:04.646157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5e985-d20a-43b5-9c65-e0b5e62e3016 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:04.646434 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5e985-d20a-43b5-9c65-e0b5e62e3016 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:04.646648 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfe6ae-a5be-454b-a46d-8b5ec26dc9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:04.646772 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5e985-d20a-43b5-9c65-e0b5e62e3016 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:04.647313 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfe6ae-a5be-454b-a46d-8b5ec26dc9bd None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:04.647542 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfe6ae-a5be-454b-a46d-8b5ec26dc9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:04.648321 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfe6ae-a5be-454b-a46d-8b5ec26dc9bd None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:04.648977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecfe6ae-a5be-454b-a46d-8b5ec26dc9bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:04.649276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecfe6ae-a5be-454b-a46d-8b5ec26dc9bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:04.650689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecfe6ae-a5be-454b-a46d-8b5ec26dc9bd None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:04.657871 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:04.658494 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 38/78] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:53:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:04.659304 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfe6ae-a5be-454b-a46d-8b5ec26dc9bd None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:04.721332 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5e985-d20a-43b5-9c65-e0b5e62e3016 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:04.722773 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5e985-d20a-43b5-9c65-e0b5e62e3016 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:04.759828 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfe6ae-a5be-454b-a46d-8b5ec26dc9bd None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:04.769766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:04.794816 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb2289-dc9e-4650-8f7b-ef550766fac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:04.795443 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb2289-dc9e-4650-8f7b-ef550766fac3 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:04.795728 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb2289-dc9e-4650-8f7b-ef550766fac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:04.796005 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb2289-dc9e-4650-8f7b-ef550766fac3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:04.797132 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bb2289-dc9e-4650-8f7b-ef550766fac3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:04.798624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bb2289-dc9e-4650-8f7b-ef550766fac3 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:04.802018 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 39/79] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:53:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:04.802018 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb2289-dc9e-4650-8f7b-ef550766fac3 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:04.820724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 41/80] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:04.821558 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5e985-d20a-43b5-9c65-e0b5e62e3016 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:04.861149 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:04.902465 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb2289-dc9e-4650-8f7b-ef550766fac3 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:04.917977 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5e985-d20a-43b5-9c65-e0b5e62e3016 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:04.935295 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666a48a-826d-4541-b824-346e12e4dd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:04.935905 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666a48a-826d-4541-b824-346e12e4dd82 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:04.936197 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666a48a-826d-4541-b824-346e12e4dd82 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:04.936475 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666a48a-826d-4541-b824-346e12e4dd82 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:04.937134 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666a48a-826d-4541-b824-346e12e4dd82 None admin] RBAC: Adding query filter params (name=member) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:04.937441 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666a48a-826d-4541-b824-346e12e4dd82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:04.938854 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666a48a-826d-4541-b824-346e12e4dd82 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:04.947457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 40/81] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:04.949560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666a48a-826d-4541-b824-346e12e4dd82 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:04.950941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba0477-567b-4ceb-b959-20212c2baa1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:04.951552 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba0477-567b-4ceb-b959-20212c2baa1c None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:04.951828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba0477-567b-4ceb-b959-20212c2baa1c None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:04.952188 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba0477-567b-4ceb-b959-20212c2baa1c None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:04.952852 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba0477-567b-4ceb-b959-20212c2baa1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:04.953148 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba0477-567b-4ceb-b959-20212c2baa1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:04.954672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba0477-567b-4ceb-b959-20212c2baa1c None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:04.964294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 42/82] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:04.964847 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba0477-567b-4ceb-b959-20212c2baa1c None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:04.994687 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:05.051927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666a48a-826d-4541-b824-346e12e4dd82 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:05.062230 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba0477-567b-4ceb-b959-20212c2baa1c None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:05.063691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:05.085178 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1b79b-d23f-4e38-9728-bcb219d1ef71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:05.085864 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1b79b-d23f-4e38-9728-bcb219d1ef71 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:05.086177 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1b79b-d23f-4e38-9728-bcb219d1ef71 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:05.086455 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1b79b-d23f-4e38-9728-bcb219d1ef71 None admin] PATH_INFO: `/v3/users/77fa2fe74be84241a0d5150ae86101fb` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:05.087578 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1b79b-d23f-4e38-9728-bcb219d1ef71 None admin] RBAC: Authorizing `identity:get_user(user_id=77fa2fe74be84241a0d5150ae86101fb)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:05.089039 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1b79b-d23f-4e38-9728-bcb219d1ef71 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:05.090650 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 41/83] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:04 2022] GET /identity/v3/users/77fa2fe74be84241a0d5150ae86101fb => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:05.091410 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1b79b-d23f-4e38-9728-bcb219d1ef71 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:05.099888 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d86869-1f69-47f0-af96-76dd71293a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:05.100495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d86869-1f69-47f0-af96-76dd71293a69 None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:05.100773 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d86869-1f69-47f0-af96-76dd71293a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:05.101078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d86869-1f69-47f0-af96-76dd71293a69 None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:05.101715 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d86869-1f69-47f0-af96-76dd71293a69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:05.103188 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d86869-1f69-47f0-af96-76dd71293a69 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:05.129864 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 43/84] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 14 01:53:04 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:53:05.130768 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d86869-1f69-47f0-af96-76dd71293a69 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:05.198852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1b79b-d23f-4e38-9728-bcb219d1ef71 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:05.209590 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:05.242820 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efafcc0e-392c-4ea6-b598-239746d73684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:05.243486 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafcc0e-392c-4ea6-b598-239746d73684 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:05.243771 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafcc0e-392c-4ea6-b598-239746d73684 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:05.244047 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafcc0e-392c-4ea6-b598-239746d73684 None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:05.245304 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efafcc0e-392c-4ea6-b598-239746d73684 None admin] RBAC: Authorizing `identity:get_project(project_id=f8542c9d54464124a730d0ce2363e406)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:05.246768 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efafcc0e-392c-4ea6-b598-239746d73684 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:05.248522 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 42/85] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:05 2022] GET /identity/v3/projects/f8542c9d54464124a730d0ce2363e406 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:05.249290 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efafcc0e-392c-4ea6-b598-239746d73684 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:05.259613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d86869-1f69-47f0-af96-76dd71293a69 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:05.297277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476188d-9e5c-4429-9af9-a8270696b7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:05.298471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476188d-9e5c-4429-9af9-a8270696b7a5 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:05.298471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476188d-9e5c-4429-9af9-a8270696b7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:05.298471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476188d-9e5c-4429-9af9-a8270696b7a5 None admin] PATH_INFO: `/v3/users/77fa2fe74be84241a0d5150ae86101fb` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:05.299620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476188d-9e5c-4429-9af9-a8270696b7a5 None admin] RBAC: Authorizing `identity:get_user(user_id=77fa2fe74be84241a0d5150ae86101fb)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:05.301064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476188d-9e5c-4429-9af9-a8270696b7a5 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:05.302673 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 44/86] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:05 2022] GET /identity/v3/users/77fa2fe74be84241a0d5150ae86101fb => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:05.303424 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476188d-9e5c-4429-9af9-a8270696b7a5 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:05.335917 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:05.370071 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efafcc0e-392c-4ea6-b598-239746d73684 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:05.417619 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf0ce0-b576-44ec-b215-ff8aa01848c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:05.418373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf0ce0-b576-44ec-b215-ff8aa01848c5 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:05.418661 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf0ce0-b576-44ec-b215-ff8aa01848c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:05.418940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf0ce0-b576-44ec-b215-ff8aa01848c5 None admin] PATH_INFO: `/v3/users/25374d78618f4e32be8780901a31679f` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:05.421283 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf0ce0-b576-44ec-b215-ff8aa01848c5 None admin] RBAC: Authorizing `identity:get_user(user_id=25374d78618f4e32be8780901a31679f)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:05.422927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf0ce0-b576-44ec-b215-ff8aa01848c5 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:05.424898 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 43/87] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:05 2022] GET /identity/v3/users/25374d78618f4e32be8780901a31679f => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:05.425390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf0ce0-b576-44ec-b215-ff8aa01848c5 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:05.439311 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476188d-9e5c-4429-9af9-a8270696b7a5 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:05.482857 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:05.483605 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad42f2-6bdf-4768-a5d9-9722c16f65a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:05.484365 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad42f2-6bdf-4768-a5d9-9722c16f65a8 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:05.484802 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad42f2-6bdf-4768-a5d9-9722c16f65a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:05.485102 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad42f2-6bdf-4768-a5d9-9722c16f65a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:05.547177 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf0ce0-b576-44ec-b215-ff8aa01848c5 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:05.560886 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad42f2-6bdf-4768-a5d9-9722c16f65a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:05.562604 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad42f2-6bdf-4768-a5d9-9722c16f65a8 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:05.584505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718c0ad-b566-4f65-84d5-a7c8e7782a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:05.585138 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718c0ad-b566-4f65-84d5-a7c8e7782a7e None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:05.585457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718c0ad-b566-4f65-84d5-a7c8e7782a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:05.585738 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718c0ad-b566-4f65-84d5-a7c8e7782a7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:05.586544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718c0ad-b566-4f65-84d5-a7c8e7782a7e None admin] RBAC: Adding query filter params (role.id=934ee0e8483c496cb8813eccbe0ebf55, scope.project.id=f8542c9d54464124a730d0ce2363e406, user.id=77fa2fe74be84241a0d5150ae86101fb) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:05.586860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718c0ad-b566-4f65-84d5-a7c8e7782a7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:05.588671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718c0ad-b566-4f65-84d5-a7c8e7782a7e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:05.590254 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:05.596038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 44/88] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:53:05 2022] GET /identity/v3/role_assignments?user.id=77fa2fe74be84241a0d5150ae86101fb&scope.project.id=f8542c9d54464124a730d0ce2363e406&role.id=934ee0e8483c496cb8813eccbe0ebf55 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:05.596702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718c0ad-b566-4f65-84d5-a7c8e7782a7e None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:05.706259 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718c0ad-b566-4f65-84d5-a7c8e7782a7e None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:05.716778 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:05.745931 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44916048-c863-4f75-ab7c-1fd317859f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:05.746746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44916048-c863-4f75-ab7c-1fd317859f2d None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:05.747099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44916048-c863-4f75-ab7c-1fd317859f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:05.747455 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44916048-c863-4f75-ab7c-1fd317859f2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:05.751939 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 45/89] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:05.752841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad42f2-6bdf-4768-a5d9-9722c16f65a8 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:05.830387 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44916048-c863-4f75-ab7c-1fd317859f2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:05.838015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44916048-c863-4f75-ab7c-1fd317859f2d None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:05.958813 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad42f2-6bdf-4768-a5d9-9722c16f65a8 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:05.981429 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 45/90] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:05.982153 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44916048-c863-4f75-ab7c-1fd317859f2d None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:05.997486 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:06.011019 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:06.029374 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95374b0b-fb27-469d-8988-79db14de535f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:06.030018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95374b0b-fb27-469d-8988-79db14de535f None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:06.030298 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95374b0b-fb27-469d-8988-79db14de535f None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:06.030568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95374b0b-fb27-469d-8988-79db14de535f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:06.147376 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44916048-c863-4f75-ab7c-1fd317859f2d None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:06.169728 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95374b0b-fb27-469d-8988-79db14de535f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:06.171435 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95374b0b-fb27-469d-8988-79db14de535f None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:06.188391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935be91c-3c62-4ccd-b120-09f9d11f93a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:06.189032 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935be91c-3c62-4ccd-b120-09f9d11f93a9 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:06.189313 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935be91c-3c62-4ccd-b120-09f9d11f93a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:06.189583 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935be91c-3c62-4ccd-b120-09f9d11f93a9 None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:06.191846 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935be91c-3c62-4ccd-b120-09f9d11f93a9 None admin] RBAC: Authorizing `identity:get_project(project_id=f8542c9d54464124a730d0ce2363e406)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:06.193293 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935be91c-3c62-4ccd-b120-09f9d11f93a9 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:06.197099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 46/91] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:05 2022] GET /identity/v3/projects/f8542c9d54464124a730d0ce2363e406 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:06.197909 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935be91c-3c62-4ccd-b120-09f9d11f93a9 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:06.304467 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:06.342484 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 46/92] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:06.343242 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95374b0b-fb27-469d-8988-79db14de535f None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:06.364131 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935be91c-3c62-4ccd-b120-09f9d11f93a9 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:06.373346 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:06.455654 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85edab99-99e0-4f82-b9eb-34e9fd72a532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:06.456280 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edab99-99e0-4f82-b9eb-34e9fd72a532 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:06.456471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edab99-99e0-4f82-b9eb-34e9fd72a532 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:06.456745 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edab99-99e0-4f82-b9eb-34e9fd72a532 None admin] PATH_INFO: `/v3/projects/2dafd3962087427ba4d7c7560d5fa88c` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:06.462655 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85edab99-99e0-4f82-b9eb-34e9fd72a532 None admin] RBAC: Authorizing `identity:get_project(project_id=2dafd3962087427ba4d7c7560d5fa88c)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:06.464136 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85edab99-99e0-4f82-b9eb-34e9fd72a532 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:06.466089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 47/93] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:06 2022] GET /identity/v3/projects/2dafd3962087427ba4d7c7560d5fa88c => generated 312 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:06.466491 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95374b0b-fb27-469d-8988-79db14de535f None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:06.466858 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85edab99-99e0-4f82-b9eb-34e9fd72a532 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:06.517808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4473d73-91bd-4082-a45e-d9aa6f1b97af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:06.518473 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4473d73-91bd-4082-a45e-d9aa6f1b97af None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:06.518749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4473d73-91bd-4082-a45e-d9aa6f1b97af None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:06.519019 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4473d73-91bd-4082-a45e-d9aa6f1b97af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:06.522287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4473d73-91bd-4082-a45e-d9aa6f1b97af None admin] RBAC: Adding query filter params (role.id=e842ac50986946a4971a51a06a39e16b, scope.project.id=f8542c9d54464124a730d0ce2363e406, user.id=77fa2fe74be84241a0d5150ae86101fb) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:06.523375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4473d73-91bd-4082-a45e-d9aa6f1b97af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:06.524847 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4473d73-91bd-4082-a45e-d9aa6f1b97af None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:06.544861 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 47/94] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:53:06 2022] GET /identity/v3/role_assignments?user.id=77fa2fe74be84241a0d5150ae86101fb&scope.project.id=f8542c9d54464124a730d0ce2363e406&role.id=e842ac50986946a4971a51a06a39e16b => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:06.546018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4473d73-91bd-4082-a45e-d9aa6f1b97af None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:06.685618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85edab99-99e0-4f82-b9eb-34e9fd72a532 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:06.763484 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909064cc-e863-48dc-b72c-f2edb0d4de32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:06.764053 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909064cc-e863-48dc-b72c-f2edb0d4de32 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:06.764332 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909064cc-e863-48dc-b72c-f2edb0d4de32 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:06.764604 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909064cc-e863-48dc-b72c-f2edb0d4de32 None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:06.765937 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909064cc-e863-48dc-b72c-f2edb0d4de32 None admin] RBAC: Authorizing `identity:get_project(project_id=f8542c9d54464124a730d0ce2363e406)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:06.767439 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909064cc-e863-48dc-b72c-f2edb0d4de32 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:06.769260 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 48/95] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:06 2022] GET /identity/v3/projects/f8542c9d54464124a730d0ce2363e406 => generated 298 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:06.777438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:06.777764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909064cc-e863-48dc-b72c-f2edb0d4de32 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:06.798304 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4473d73-91bd-4082-a45e-d9aa6f1b97af None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:06.830236 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:06.891949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d32ac-8332-4ff4-bb8c-7334ebbfd344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:06.892573 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d32ac-8332-4ff4-bb8c-7334ebbfd344 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:06.892854 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d32ac-8332-4ff4-bb8c-7334ebbfd344 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:06.893128 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d32ac-8332-4ff4-bb8c-7334ebbfd344 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:06.893858 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d32ac-8332-4ff4-bb8c-7334ebbfd344 None admin] RBAC: Adding query filter params (role.id=934ee0e8483c496cb8813eccbe0ebf55, scope.project.id=2dafd3962087427ba4d7c7560d5fa88c, user.id=77fa2fe74be84241a0d5150ae86101fb) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:06.894190 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d32ac-8332-4ff4-bb8c-7334ebbfd344 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:06.895651 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d32ac-8332-4ff4-bb8c-7334ebbfd344 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:06.906322 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 48/96] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:53:06 2022] GET /identity/v3/role_assignments?user.id=77fa2fe74be84241a0d5150ae86101fb&scope.project.id=2dafd3962087427ba4d7c7560d5fa88c&role.id=934ee0e8483c496cb8813eccbe0ebf55 => generated 259 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:06.975088 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909064cc-e863-48dc-b72c-f2edb0d4de32 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:07.019681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e17b6-d283-4180-8e88-5b4ad50e6e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:07.020220 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e17b6-d283-4180-8e88-5b4ad50e6e20 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:07.020499 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e17b6-d283-4180-8e88-5b4ad50e6e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:07.020771 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e17b6-d283-4180-8e88-5b4ad50e6e20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:07.021498 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e17b6-d283-4180-8e88-5b4ad50e6e20 None admin] RBAC: Adding query filter params (role.id=aa420c2fb1ab4bb3b05b004a4d89779a, scope.project.id=f8542c9d54464124a730d0ce2363e406, user.id=25374d78618f4e32be8780901a31679f) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:07.021807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e17b6-d283-4180-8e88-5b4ad50e6e20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:07.023396 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e17b6-d283-4180-8e88-5b4ad50e6e20 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:07.029615 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 49/97] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:53:06 2022] GET /identity/v3/role_assignments?user.id=25374d78618f4e32be8780901a31679f&scope.project.id=f8542c9d54464124a730d0ce2363e406&role.id=aa420c2fb1ab4bb3b05b004a4d89779a => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:08.144726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc295bf-ece3-47e1-8b2f-6effb8a63b33 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:08.145735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc295bf-ece3-47e1-8b2f-6effb8a63b33 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:08.146285 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc295bf-ece3-47e1-8b2f-6effb8a63b33 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:08.148321 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 49/98] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:08.153935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a044956-c2b5-4c06-94eb-2fe74c81b2cb None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:08.154781 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a044956-c2b5-4c06-94eb-2fe74c81b2cb None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:08.155254 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a044956-c2b5-4c06-94eb-2fe74c81b2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:08.175358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-4a044956-c2b5-4c06-94eb-2fe74c81b2cb None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:08.176908 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-4a044956-c2b5-4c06-94eb-2fe74c81b2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:08.200501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae9537-1a8a-408b-ba4f-69a0afd323bd None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:08.201104 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae9537-1a8a-408b-ba4f-69a0afd323bd None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:08.201572 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae9537-1a8a-408b-ba4f-69a0afd323bd None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:08.203137 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 50/99] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:53:08.209174 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f97f4-f4d1-4f7c-8610-01bc8430650a None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:08.209670 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f97f4-f4d1-4f7c-8610-01bc8430650a None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:08.210660 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f97f4-f4d1-4f7c-8610-01bc8430650a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:08.229613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-9a7f97f4-f4d1-4f7c-8610-01bc8430650a None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:08.231471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-9a7f97f4-f4d1-4f7c-8610-01bc8430650a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:08.280483 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 50/100] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:08.355365 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 51/101] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:53:08.450500 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2aea76-e9dd-4e61-acad-8c2c305f70c4 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:08.450783 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2aea76-e9dd-4e61-acad-8c2c305f70c4 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:08.450990 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2aea76-e9dd-4e61-acad-8c2c305f70c4 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:08.452349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 51/102] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:08.457412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a2d93-2326-4446-8968-1ac6bb66343b None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:08.457698 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a2d93-2326-4446-8968-1ac6bb66343b None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:08.457973 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a2d93-2326-4446-8968-1ac6bb66343b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:08.486958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-a44a2d93-2326-4446-8968-1ac6bb66343b None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:08.488130 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-a44a2d93-2326-4446-8968-1ac6bb66343b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:08.588040 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 52/103] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:08.701487 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2aea76-e9dd-4e61-acad-8c2c305f70c4 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:08.710635 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:08.760901 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a2d93-2326-4446-8968-1ac6bb66343b None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:08.779303 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:08.875921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2aea76-e9dd-4e61-acad-8c2c305f70c4 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:08.928554 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a2d93-2326-4446-8968-1ac6bb66343b None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:08.942809 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bc0af-2496-4ed8-9565-524409ad4c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:08.943845 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bc0af-2496-4ed8-9565-524409ad4c2a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:08.944379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bc0af-2496-4ed8-9565-524409ad4c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:08.944847 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bc0af-2496-4ed8-9565-524409ad4c2a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:08.946252 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bc0af-2496-4ed8-9565-524409ad4c2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:08.949455 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bc0af-2496-4ed8-9565-524409ad4c2a None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:08.958537 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 52/104] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:53:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:08.961471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bc0af-2496-4ed8-9565-524409ad4c2a None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:08.999963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c13de-fbb4-4b95-8000-49cb47e4b320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:09.000930 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c13de-fbb4-4b95-8000-49cb47e4b320 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:09.001422 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c13de-fbb4-4b95-8000-49cb47e4b320 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:09.001903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c13de-fbb4-4b95-8000-49cb47e4b320 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:09.024504 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c13de-fbb4-4b95-8000-49cb47e4b320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:09.034643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-af7c13de-fbb4-4b95-8000-49cb47e4b320 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:53:09.034643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:09.034643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:09.034643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:09.034643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:09.034643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:09.034643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:09.034643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:09.034643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:09.034643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:09.034643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:09.034643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:09.034643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:09.034643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:09.034643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:09.034643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:09.035549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:09.035549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:09.035549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:09.035549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:09.035549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:09.035549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:09.035549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:09.035549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:09.035549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:09.035549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:09.035549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:09.035549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:09.035549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:09.035549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:09.035549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:09.035549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:09.036336 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:09.036336 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:09.036336 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:09.036336 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:09.036336 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:09.036336 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:09.036336 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:09.036336 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:09.036336 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:09.036336 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:09.036336 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:53:09.036336 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:09.039621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 53/105] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:53:09.043355 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c13de-fbb4-4b95-8000-49cb47e4b320 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:09.098090 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:09.160545 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:09.161892 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bc0af-2496-4ed8-9565-524409ad4c2a None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:09.191303 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c13de-fbb4-4b95-8000-49cb47e4b320 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:09.230017 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208e248-0bab-44a5-b9e8-a1eee3a2fbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:09.231299 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208e248-0bab-44a5-b9e8-a1eee3a2fbbb None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:09.231783 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208e248-0bab-44a5-b9e8-a1eee3a2fbbb None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:09.232281 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208e248-0bab-44a5-b9e8-a1eee3a2fbbb None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:09.233268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208e248-0bab-44a5-b9e8-a1eee3a2fbbb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:09.234993 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208e248-0bab-44a5-b9e8-a1eee3a2fbbb None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:09.266548 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 54/106] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 14 01:53:09 2022] POST /identity/v3/projects => generated 302 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:53:09.267766 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208e248-0bab-44a5-b9e8-a1eee3a2fbbb None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:09.282706 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83680a9-43c7-40d7-86e5-9cc87fd492b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:09.283356 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83680a9-43c7-40d7-86e5-9cc87fd492b1 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:09.283633 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83680a9-43c7-40d7-86e5-9cc87fd492b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:09.283904 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83680a9-43c7-40d7-86e5-9cc87fd492b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:09.326302 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83680a9-43c7-40d7-86e5-9cc87fd492b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:09.333747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-e83680a9-43c7-40d7-86e5-9cc87fd492b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:09.333747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:09.333747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:09.333747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:09.333747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:09.333747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:09.333747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:09.333747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:09.333747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:09.333747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:09.333747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:09.333747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:09.333747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:09.333747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:09.333747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:09.333747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:09.334732 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:09.334732 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:09.334732 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:09.334732 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:09.334732 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:09.334732 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:09.334732 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:09.334732 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:09.334732 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:09.334732 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:09.334732 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:09.334732 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:09.334732 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:09.334732 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:09.334732 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:09.334732 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:09.335627 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:09.335627 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:09.335627 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:09.335627 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:09.335627 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:09.335627 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:09.335627 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:09.335627 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:09.335627 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:09.335627 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:09.335627 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:09.335627 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:09.338543 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 53/107] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:09.360612 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83680a9-43c7-40d7-86e5-9cc87fd492b1 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:09.461742 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208e248-0bab-44a5-b9e8-a1eee3a2fbbb None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:09.507328 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:09.520940 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3790a-b70b-4b1f-9cf5-70d76d32de57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:09.521598 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3790a-b70b-4b1f-9cf5-70d76d32de57 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:09.521886 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3790a-b70b-4b1f-9cf5-70d76d32de57 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:09.522197 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3790a-b70b-4b1f-9cf5-70d76d32de57 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:09.523428 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf3790a-b70b-4b1f-9cf5-70d76d32de57 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:09.523744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf3790a-b70b-4b1f-9cf5-70d76d32de57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:09.525155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf3790a-b70b-4b1f-9cf5-70d76d32de57 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:09.540199 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 55/108] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:09.547817 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3790a-b70b-4b1f-9cf5-70d76d32de57 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:09.636308 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83680a9-43c7-40d7-86e5-9cc87fd492b1 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:09.684573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd8039-b0b4-443c-962a-962fd19234a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:09.685194 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd8039-b0b4-443c-962a-962fd19234a3 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:09.685475 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd8039-b0b4-443c-962a-962fd19234a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:09.685751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd8039-b0b4-443c-962a-962fd19234a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:09.687573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd8039-b0b4-443c-962a-962fd19234a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:09.687895 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd8039-b0b4-443c-962a-962fd19234a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:09.689290 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd8039-b0b4-443c-962a-962fd19234a3 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:09.691034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:09.708494 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 54/109] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:09.735173 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3790a-b70b-4b1f-9cf5-70d76d32de57 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:09.740790 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd8039-b0b4-443c-962a-962fd19234a3 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:09.796606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce03aee-1fc2-4b7b-acbd-8cbf3a7ae327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:09.797668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce03aee-1fc2-4b7b-acbd-8cbf3a7ae327 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:09.798190 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce03aee-1fc2-4b7b-acbd-8cbf3a7ae327 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:09.798659 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce03aee-1fc2-4b7b-acbd-8cbf3a7ae327 None admin] PATH_INFO: `/v3/users/11f11d9b47f746229df9aeeead73a6ad` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:09.816652 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce03aee-1fc2-4b7b-acbd-8cbf3a7ae327 None admin] RBAC: Authorizing `identity:get_user(user_id=11f11d9b47f746229df9aeeead73a6ad)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:09.818603 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce03aee-1fc2-4b7b-acbd-8cbf3a7ae327 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:09.820770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 56/110] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:09 2022] GET /identity/v3/users/11f11d9b47f746229df9aeeead73a6ad => generated 289 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:09.822369 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b4e5c-555c-422b-a2cf-48eb2b71d3fb None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:09.822915 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b4e5c-555c-422b-a2cf-48eb2b71d3fb None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:09.823475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b4e5c-555c-422b-a2cf-48eb2b71d3fb None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:09.824887 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 57/111] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:09.829949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b4e5c-555c-422b-a2cf-48eb2b71d3fb None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:09.869974 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd8039-b0b4-443c-962a-962fd19234a3 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:09.908779 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3b742-4223-46ff-92e4-a99f5232d271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:09.909670 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3b742-4223-46ff-92e4-a99f5232d271 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:09.910054 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3b742-4223-46ff-92e4-a99f5232d271 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:09.910567 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:09.910671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3b742-4223-46ff-92e4-a99f5232d271 None admin] PATH_INFO: `/v3/users/77fa2fe74be84241a0d5150ae86101fb` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:09.912043 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3b742-4223-46ff-92e4-a99f5232d271 None admin] RBAC: Authorizing `identity:get_user(user_id=77fa2fe74be84241a0d5150ae86101fb)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:09.913548 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3b742-4223-46ff-92e4-a99f5232d271 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:09.916040 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 55/112] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:09 2022] GET /identity/v3/users/77fa2fe74be84241a0d5150ae86101fb => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:09.917196 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9a718-4e25-4001-b271-2b145dc628eb None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:09.917534 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9a718-4e25-4001-b271-2b145dc628eb None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:09.917760 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9a718-4e25-4001-b271-2b145dc628eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:09.939856 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-efc9a718-4e25-4001-b271-2b145dc628eb None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:09.941069 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-efc9a718-4e25-4001-b271-2b145dc628eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:09.956036 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b4e5c-555c-422b-a2cf-48eb2b71d3fb None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:09.997715 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92100659-49e8-4938-be4c-7997b744092e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:09.998472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92100659-49e8-4938-be4c-7997b744092e None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:09.998806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92100659-49e8-4938-be4c-7997b744092e None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:09.999086 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92100659-49e8-4938-be4c-7997b744092e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:10.014147 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:10.023614 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 56/113] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:10.025121 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369a0f0-f70d-435b-90f2-444734e49385 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:10.025484 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369a0f0-f70d-435b-90f2-444734e49385 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:10.025884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369a0f0-f70d-435b-90f2-444734e49385 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:10.027290 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 57/114] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:10.028116 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369a0f0-f70d-435b-90f2-444734e49385 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:10.098734 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92100659-49e8-4938-be4c-7997b744092e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:10.101208 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92100659-49e8-4938-be4c-7997b744092e None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:10.144751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369a0f0-f70d-435b-90f2-444734e49385 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:10.157653 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:10.218592 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534ea2a9-7dbf-4031-b8c4-e9448457c7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:10.220982 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ea2a9-7dbf-4031-b8c4-e9448457c7e4 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:10.221527 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ea2a9-7dbf-4031-b8c4-e9448457c7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:10.225148 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ea2a9-7dbf-4031-b8c4-e9448457c7e4 None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:10.227139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534ea2a9-7dbf-4031-b8c4-e9448457c7e4 None admin] RBAC: Authorizing `identity:get_project(project_id=f8542c9d54464124a730d0ce2363e406)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:10.228938 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534ea2a9-7dbf-4031-b8c4-e9448457c7e4 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:10.232808 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 58/115] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:10 2022] GET /identity/v3/projects/f8542c9d54464124a730d0ce2363e406 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:10.237453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a5363-0401-42c9-959e-7f99994d91de None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:10.238330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a5363-0401-42c9-959e-7f99994d91de None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:10.239046 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a5363-0401-42c9-959e-7f99994d91de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:10.280489 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 58/116] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 01:53:10.282139 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92100659-49e8-4938-be4c-7997b744092e None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:10.282750 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-386a5363-0401-42c9-959e-7f99994d91de None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:10.283974 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-386a5363-0401-42c9-959e-7f99994d91de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:10.399280 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:10.408077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92100659-49e8-4938-be4c-7997b744092e None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:10.418005 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 59/117] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:10.418005 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a5363-0401-42c9-959e-7f99994d91de None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:10.462700 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede09ae-71a0-4a93-855a-147e123da6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:10.463655 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede09ae-71a0-4a93-855a-147e123da6d7 None admin] REQUEST_METHOD: `PUT` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:10.463944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede09ae-71a0-4a93-855a-147e123da6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:10.464258 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede09ae-71a0-4a93-855a-147e123da6d7 None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406/users/77fa2fe74be84241a0d5150ae86101fb/roles/934ee0e8483c496cb8813eccbe0ebf55` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:10.466907 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede09ae-71a0-4a93-855a-147e123da6d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8542c9d54464124a730d0ce2363e406, user_id=77fa2fe74be84241a0d5150ae86101fb, role_id=934ee0e8483c496cb8813eccbe0ebf55)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:10.468411 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede09ae-71a0-4a93-855a-147e123da6d7 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:10.490902 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 59/118] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:53:10 2022] PUT /identity/v3/projects/f8542c9d54464124a730d0ce2363e406/users/77fa2fe74be84241a0d5150ae86101fb/roles/934ee0e8483c496cb8813eccbe0ebf55 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:53:10.494958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede09ae-71a0-4a93-855a-147e123da6d7 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:10.516877 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:10.542227 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a5363-0401-42c9-959e-7f99994d91de None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:10.608383 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a49af7-1e8a-4f14-8fb4-a0003eb49d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:10.609046 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a49af7-1e8a-4f14-8fb4-a0003eb49d57 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:10.609323 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a49af7-1e8a-4f14-8fb4-a0003eb49d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:10.609597 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a49af7-1e8a-4f14-8fb4-a0003eb49d57 None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:10.611048 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a49af7-1e8a-4f14-8fb4-a0003eb49d57 None admin] RBAC: Authorizing `identity:get_project(project_id=f8542c9d54464124a730d0ce2363e406)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:10.612539 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a49af7-1e8a-4f14-8fb4-a0003eb49d57 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:10.621299 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 60/119] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:10 2022] GET /identity/v3/projects/f8542c9d54464124a730d0ce2363e406 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:10.623554 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b7772-d139-4dcd-baf3-a9990a8cf67e None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:10.623840 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b7772-d139-4dcd-baf3-a9990a8cf67e None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:10.624113 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b7772-d139-4dcd-baf3-a9990a8cf67e None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:10.625652 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 61/120] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:10.631806 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b7772-d139-4dcd-baf3-a9990a8cf67e None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:10.713799 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede09ae-71a0-4a93-855a-147e123da6d7 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:10.745165 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:10.764332 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3907f-9f8e-43b9-b92f-2e1d7ff2f3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:10.764974 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3907f-9f8e-43b9-b92f-2e1d7ff2f3ca None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:10.765252 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3907f-9f8e-43b9-b92f-2e1d7ff2f3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:10.765522 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3907f-9f8e-43b9-b92f-2e1d7ff2f3ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:10.777825 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3907f-9f8e-43b9-b92f-2e1d7ff2f3ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:10.785591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-bcb3907f-9f8e-43b9-b92f-2e1d7ff2f3ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:10.785591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:10.785591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:10.785591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:10.785591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:10.785591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:10.785591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:10.785591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:10.785591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:10.785591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:10.785591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:10.785591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:10.785591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:10.785591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:10.785591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:10.785591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:10.786526 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:10.786526 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:10.786526 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:10.786526 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:10.786526 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:10.786526 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:10.786526 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:10.786526 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:10.786526 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:10.786526 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:10.786526 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:10.786526 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:10.786526 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:10.786526 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:10.786526 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:10.786526 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:10.787312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:10.787312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:10.787312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:10.787312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:10.787312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:10.787312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:10.787312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:10.787312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:10.787312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:10.787312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:10.787312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:10.787312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:10.787312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 60/121] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:53:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:10.789856 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d4be9-7a42-4aab-b01f-d0e229fc58ad None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:10.790949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d4be9-7a42-4aab-b01f-d0e229fc58ad None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:10.791232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d4be9-7a42-4aab-b01f-d0e229fc58ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:10.804497 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b7772-d139-4dcd-baf3-a9990a8cf67e None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:10.810456 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-075d4be9-7a42-4aab-b01f-d0e229fc58ad None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:10.811655 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-075d4be9-7a42-4aab-b01f-d0e229fc58ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:10.841811 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e061e-24a4-46a0-88bc-c4b2148596aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:10.842452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e061e-24a4-46a0-88bc-c4b2148596aa None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:10.842872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e061e-24a4-46a0-88bc-c4b2148596aa None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:10.843095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e061e-24a4-46a0-88bc-c4b2148596aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:10.844050 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e061e-24a4-46a0-88bc-c4b2148596aa None admin] RBAC: Adding query filter params (role.id=e5ce565bb4ca4bad8c14b5e76d2ae8ea, scope.project.id=f8542c9d54464124a730d0ce2363e406, user.id=11f11d9b47f746229df9aeeead73a6ad) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:10.844234 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e061e-24a4-46a0-88bc-c4b2148596aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:10.845687 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e061e-24a4-46a0-88bc-c4b2148596aa None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:10.851835 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 62/122] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:53:10 2022] GET /identity/v3/role_assignments?user.id=11f11d9b47f746229df9aeeead73a6ad&scope.project.id=f8542c9d54464124a730d0ce2363e406&role.id=e5ce565bb4ca4bad8c14b5e76d2ae8ea => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:10.852623 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e061e-24a4-46a0-88bc-c4b2148596aa None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:10.877438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:10.886561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 61/123] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:10.887715 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d4be9-7a42-4aab-b01f-d0e229fc58ad None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:11.058269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d4be9-7a42-4aab-b01f-d0e229fc58ad None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:11.121869 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a94fa-e865-40f0-a14d-c96a96ba05ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:11.122858 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a94fa-e865-40f0-a14d-c96a96ba05ee None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:11.123551 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a94fa-e865-40f0-a14d-c96a96ba05ee None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:11.123962 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a94fa-e865-40f0-a14d-c96a96ba05ee None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:11.124767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a94fa-e865-40f0-a14d-c96a96ba05ee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:11.125200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a94fa-e865-40f0-a14d-c96a96ba05ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:11.126786 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a94fa-e865-40f0-a14d-c96a96ba05ee None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:11.134030 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e061e-24a4-46a0-88bc-c4b2148596aa None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:11.140300 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 62/124] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:53:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:11.153904 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a94fa-e865-40f0-a14d-c96a96ba05ee None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:11.161600 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:11.174710 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7dd72-122d-40f0-928a-ef66116fa941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:11.175352 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7dd72-122d-40f0-928a-ef66116fa941 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:11.175633 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7dd72-122d-40f0-928a-ef66116fa941 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:11.175908 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7dd72-122d-40f0-928a-ef66116fa941 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:11.187023 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d7dd72-122d-40f0-928a-ef66116fa941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:11.192593 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-27d7dd72-122d-40f0-928a-ef66116fa941 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:11.192593 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:11.192593 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:11.192593 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:11.192593 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:11.192593 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:11.192593 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:11.192593 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:11.192593 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:11.192593 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:11.192593 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:11.192593 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:11.192593 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:11.192593 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:11.192593 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:11.192593 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:11.193469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:11.193469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:11.193469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:11.193469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:11.193469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:11.193469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:11.193469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:11.193469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:11.193469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:11.193469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:11.193469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:11.193469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:11.193469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:11.193469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:11.193469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:11.193469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:11.195070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:11.195070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:11.195070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:11.195070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:11.195070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:11.195070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:11.195070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:11.195070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:11.195070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:11.195070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:11.195070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:11.195070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:11.195070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 63/125] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:11.199615 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7dd72-122d-40f0-928a-ef66116fa941 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:11.303308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:11.321484 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7dd72-122d-40f0-928a-ef66116fa941 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:11.375689 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a94fa-e865-40f0-a14d-c96a96ba05ee None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:11.388492 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1880c-1b69-4a94-a37b-3890482459c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:11.388847 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1880c-1b69-4a94-a37b-3890482459c6 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:11.389129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1880c-1b69-4a94-a37b-3890482459c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:11.389402 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1880c-1b69-4a94-a37b-3890482459c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:11.391893 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1880c-1b69-4a94-a37b-3890482459c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:11.391893 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1880c-1b69-4a94-a37b-3890482459c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:11.391893 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1880c-1b69-4a94-a37b-3890482459c6 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:11.401156 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 64/126] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:11.406863 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1880c-1b69-4a94-a37b-3890482459c6 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:11.442358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c44d6-c142-42a9-914c-a81827191014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:11.443133 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c44d6-c142-42a9-914c-a81827191014 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:11.443427 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c44d6-c142-42a9-914c-a81827191014 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:11.443706 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c44d6-c142-42a9-914c-a81827191014 None admin] PATH_INFO: `/v3/users/77fa2fe74be84241a0d5150ae86101fb` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:11.445070 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c44d6-c142-42a9-914c-a81827191014 None admin] RBAC: Authorizing `identity:get_user(user_id=77fa2fe74be84241a0d5150ae86101fb)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:11.446667 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c44d6-c142-42a9-914c-a81827191014 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:11.448710 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 63/127] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:11 2022] GET /identity/v3/users/77fa2fe74be84241a0d5150ae86101fb => generated 275 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:11.449660 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c44d6-c142-42a9-914c-a81827191014 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:11.465837 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:11.522020 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1880c-1b69-4a94-a37b-3890482459c6 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:11.567503 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b8c0c-13c3-438c-a590-76d1d2448a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:11.568193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b8c0c-13c3-438c-a590-76d1d2448a86 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:11.568476 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b8c0c-13c3-438c-a590-76d1d2448a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:11.568746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b8c0c-13c3-438c-a590-76d1d2448a86 None admin] PATH_INFO: `/v3/users/77fa2fe74be84241a0d5150ae86101fb` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:11.570653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6b8c0c-13c3-438c-a590-76d1d2448a86 None admin] RBAC: Authorizing `identity:get_user(user_id=77fa2fe74be84241a0d5150ae86101fb)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:11.572167 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6b8c0c-13c3-438c-a590-76d1d2448a86 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:11.573684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 65/128] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:11 2022] GET /identity/v3/users/77fa2fe74be84241a0d5150ae86101fb => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:11.574473 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b8c0c-13c3-438c-a590-76d1d2448a86 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:11.574577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c44d6-c142-42a9-914c-a81827191014 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:11.603015 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:11.613724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad988db0-c2a8-4b69-bd89-893c80b0cdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:11.614494 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad988db0-c2a8-4b69-bd89-893c80b0cdac None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:11.614828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad988db0-c2a8-4b69-bd89-893c80b0cdac None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:11.615110 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad988db0-c2a8-4b69-bd89-893c80b0cdac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:11.627660 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad988db0-c2a8-4b69-bd89-893c80b0cdac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:11.634600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-ad988db0-c2a8-4b69-bd89-893c80b0cdac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:53:11.634600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:11.634600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:11.634600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:11.634600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:11.634600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:11.634600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:11.634600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:11.634600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:11.634600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:11.634600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:11.634600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:11.634600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:11.634600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:11.634600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:11.634600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:11.635613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:11.635613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:11.635613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:11.635613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:11.635613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:11.635613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:11.635613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:11.635613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:11.635613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:11.635613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:11.635613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:11.635613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:11.635613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:11.635613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:11.635613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:11.635613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:11.637213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:11.637213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:11.637213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:11.637213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:11.637213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:11.637213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:11.637213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:11.637213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:11.637213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:11.637213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:11.637213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:53:11.637213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:11.637213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 64/129] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:53:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:11.637213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad988db0-c2a8-4b69-bd89-893c80b0cdac None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:11.683268 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b8c0c-13c3-438c-a590-76d1d2448a86 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:11.697573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:11.719302 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b38577-8e7c-4ae2-a618-b9a329f3a161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:11.719935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b38577-8e7c-4ae2-a618-b9a329f3a161 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:11.720269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b38577-8e7c-4ae2-a618-b9a329f3a161 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:11.720550 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b38577-8e7c-4ae2-a618-b9a329f3a161 None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:11.721975 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b38577-8e7c-4ae2-a618-b9a329f3a161 None admin] RBAC: Authorizing `identity:get_project(project_id=f8542c9d54464124a730d0ce2363e406)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:11.724457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b38577-8e7c-4ae2-a618-b9a329f3a161 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:11.727255 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 66/130] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:11 2022] GET /identity/v3/projects/f8542c9d54464124a730d0ce2363e406 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:11.728245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b38577-8e7c-4ae2-a618-b9a329f3a161 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:11.753881 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad988db0-c2a8-4b69-bd89-893c80b0cdac None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:11.791709 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7c4f8-4e1d-4e4d-8108-b1cdff5b2963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:11.792615 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7c4f8-4e1d-4e4d-8108-b1cdff5b2963 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:11.792753 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7c4f8-4e1d-4e4d-8108-b1cdff5b2963 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:11.793037 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7c4f8-4e1d-4e4d-8108-b1cdff5b2963 None admin] PATH_INFO: `/v3/projects/2dafd3962087427ba4d7c7560d5fa88c` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:11.794209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b7c4f8-4e1d-4e4d-8108-b1cdff5b2963 None admin] RBAC: Authorizing `identity:get_project(project_id=2dafd3962087427ba4d7c7560d5fa88c)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:11.796129 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b7c4f8-4e1d-4e4d-8108-b1cdff5b2963 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:11.797671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 65/131] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:11 2022] GET /identity/v3/projects/2dafd3962087427ba4d7c7560d5fa88c => generated 312 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:11.802004 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7c4f8-4e1d-4e4d-8108-b1cdff5b2963 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:11.846850 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b38577-8e7c-4ae2-a618-b9a329f3a161 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:11.861123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:11.883568 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac7dcf-bab2-4d5a-aa92-77e36c97e67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:11.884453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac7dcf-bab2-4d5a-aa92-77e36c97e67a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:11.884946 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac7dcf-bab2-4d5a-aa92-77e36c97e67a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:11.885362 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac7dcf-bab2-4d5a-aa92-77e36c97e67a None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:11.886187 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac7dcf-bab2-4d5a-aa92-77e36c97e67a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:11.886630 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac7dcf-bab2-4d5a-aa92-77e36c97e67a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:11.888150 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac7dcf-bab2-4d5a-aa92-77e36c97e67a None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:11.896841 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 67/132] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:11.897866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac7dcf-bab2-4d5a-aa92-77e36c97e67a None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:11.911920 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7c4f8-4e1d-4e4d-8108-b1cdff5b2963 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:11.946353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:11.949863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194a776-e749-47a7-9d10-825935f8c110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:11.950568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194a776-e749-47a7-9d10-825935f8c110 None admin] REQUEST_METHOD: `PUT` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:11.950852 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194a776-e749-47a7-9d10-825935f8c110 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:11.951288 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194a776-e749-47a7-9d10-825935f8c110 None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406/users/77fa2fe74be84241a0d5150ae86101fb/roles/e842ac50986946a4971a51a06a39e16b` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:11.954299 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194a776-e749-47a7-9d10-825935f8c110 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8542c9d54464124a730d0ce2363e406, user_id=77fa2fe74be84241a0d5150ae86101fb, role_id=e842ac50986946a4971a51a06a39e16b)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:11.955887 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194a776-e749-47a7-9d10-825935f8c110 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:11.972083 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 66/133] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:53:11 2022] PUT /identity/v3/projects/f8542c9d54464124a730d0ce2363e406/users/77fa2fe74be84241a0d5150ae86101fb/roles/e842ac50986946a4971a51a06a39e16b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:53:11.973720 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194a776-e749-47a7-9d10-825935f8c110 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:12.044587 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac7dcf-bab2-4d5a-aa92-77e36c97e67a None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:12.130238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f6ec1-e2e6-48e1-99a4-635da4caed29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:12.130842 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f6ec1-e2e6-48e1-99a4-635da4caed29 None admin] REQUEST_METHOD: `PUT` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:12.131121 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f6ec1-e2e6-48e1-99a4-635da4caed29 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:12.131393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f6ec1-e2e6-48e1-99a4-635da4caed29 None admin] PATH_INFO: `/v3/projects/2dafd3962087427ba4d7c7560d5fa88c/users/77fa2fe74be84241a0d5150ae86101fb/roles/934ee0e8483c496cb8813eccbe0ebf55` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:12.133716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f6ec1-e2e6-48e1-99a4-635da4caed29 None admin] RBAC: Authorizing `identity:create_grant(project_id=2dafd3962087427ba4d7c7560d5fa88c, user_id=77fa2fe74be84241a0d5150ae86101fb, role_id=934ee0e8483c496cb8813eccbe0ebf55)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:12.135149 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f6ec1-e2e6-48e1-99a4-635da4caed29 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:12.170636 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 68/134] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:53:11 2022] PUT /identity/v3/projects/2dafd3962087427ba4d7c7560d5fa88c/users/77fa2fe74be84241a0d5150ae86101fb/roles/934ee0e8483c496cb8813eccbe0ebf55 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:53:12.184056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194a776-e749-47a7-9d10-825935f8c110 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:12.200815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:12.228949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0139bb-21d8-454e-909d-66b92f6b7472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:12.229715 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0139bb-21d8-454e-909d-66b92f6b7472 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:12.230265 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0139bb-21d8-454e-909d-66b92f6b7472 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:12.230265 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0139bb-21d8-454e-909d-66b92f6b7472 None admin] PATH_INFO: `/v3/users/25374d78618f4e32be8780901a31679f` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:12.231444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0139bb-21d8-454e-909d-66b92f6b7472 None admin] RBAC: Authorizing `identity:get_user(user_id=25374d78618f4e32be8780901a31679f)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:12.233758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0139bb-21d8-454e-909d-66b92f6b7472 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:12.234835 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 67/135] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:11 2022] GET /identity/v3/users/25374d78618f4e32be8780901a31679f => generated 247 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:12.247418 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f6ec1-e2e6-48e1-99a4-635da4caed29 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:12.364486 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f6ec1-e2e6-48e1-99a4-635da4caed29 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:12.367214 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:12.417347 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9eabc-ad96-4292-ac36-ac6df029ac6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:12.418022 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9eabc-ad96-4292-ac36-ac6df029ac6d None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:12.418309 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9eabc-ad96-4292-ac36-ac6df029ac6d None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:12.418585 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9eabc-ad96-4292-ac36-ac6df029ac6d None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:12.419757 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9eabc-ad96-4292-ac36-ac6df029ac6d None admin] RBAC: Authorizing `identity:get_project(project_id=f8542c9d54464124a730d0ce2363e406)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:12.421230 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9eabc-ad96-4292-ac36-ac6df029ac6d None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:12.423220 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 69/136] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:12 2022] GET /identity/v3/projects/f8542c9d54464124a730d0ce2363e406 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:12.429153 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0139bb-21d8-454e-909d-66b92f6b7472 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:12.547507 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0139bb-21d8-454e-909d-66b92f6b7472 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:12.584131 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916785f-f183-43bd-95fe-c03c19823fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:12.584942 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916785f-f183-43bd-95fe-c03c19823fa2 None admin] REQUEST_METHOD: `PUT` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:12.585229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916785f-f183-43bd-95fe-c03c19823fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:12.585559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916785f-f183-43bd-95fe-c03c19823fa2 None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406/users/25374d78618f4e32be8780901a31679f/roles/aa420c2fb1ab4bb3b05b004a4d89779a` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:12.587550 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916785f-f183-43bd-95fe-c03c19823fa2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8542c9d54464124a730d0ce2363e406, user_id=25374d78618f4e32be8780901a31679f, role_id=aa420c2fb1ab4bb3b05b004a4d89779a)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:12.589038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916785f-f183-43bd-95fe-c03c19823fa2 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:12.609606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 68/137] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:53:12 2022] PUT /identity/v3/projects/f8542c9d54464124a730d0ce2363e406/users/25374d78618f4e32be8780901a31679f/roles/aa420c2fb1ab4bb3b05b004a4d89779a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:53:12.784581 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bac4c9-730c-4e86-bcda-6bfe5c78be4b None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:12.784764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bac4c9-730c-4e86-bcda-6bfe5c78be4b None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:12.785049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bac4c9-730c-4e86-bcda-6bfe5c78be4b None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:12.786890 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 70/138] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:12.792692 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6ea9d-1b3a-4f53-97bd-e22652a46e01 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:12.792996 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6ea9d-1b3a-4f53-97bd-e22652a46e01 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:12.793253 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6ea9d-1b3a-4f53-97bd-e22652a46e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:12.820915 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-00a6ea9d-1b3a-4f53-97bd-e22652a46e01 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:12.826359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-00a6ea9d-1b3a-4f53-97bd-e22652a46e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:12.883685 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:12.932917 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 69/139] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:13.216954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bac4c9-730c-4e86-bcda-6bfe5c78be4b None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:13.334073 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bac4c9-730c-4e86-bcda-6bfe5c78be4b None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:13.377442 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16927706-404d-4be5-a6a8-72b13e7908d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:13.378905 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16927706-404d-4be5-a6a8-72b13e7908d3 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:13.379385 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16927706-404d-4be5-a6a8-72b13e7908d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:13.379849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16927706-404d-4be5-a6a8-72b13e7908d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:13.382785 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16927706-404d-4be5-a6a8-72b13e7908d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:13.387815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16927706-404d-4be5-a6a8-72b13e7908d3 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:13.387815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 71/140] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:53:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:13.395409 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6ea9d-1b3a-4f53-97bd-e22652a46e01 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:13.504907 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6ea9d-1b3a-4f53-97bd-e22652a46e01 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:13.543510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65712d8-8f62-4bd9-9f10-b576b3253e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:13.544536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65712d8-8f62-4bd9-9f10-b576b3253e71 None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:13.545012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65712d8-8f62-4bd9-9f10-b576b3253e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:13.545487 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65712d8-8f62-4bd9-9f10-b576b3253e71 None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:13.546385 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65712d8-8f62-4bd9-9f10-b576b3253e71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:13.548050 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65712d8-8f62-4bd9-9f10-b576b3253e71 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:13.574022 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 70/141] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 14 01:53:13 2022] POST /identity/v3/users => generated 283 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:53:13.788602 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622e24b-93a9-4959-a63d-a988bca8a718 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:13.788780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622e24b-93a9-4959-a63d-a988bca8a718 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:13.789081 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622e24b-93a9-4959-a63d-a988bca8a718 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:13.790597 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 72/142] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:13.803949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e386e-9891-473c-83ad-ccc05365c3ff None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:13.804577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e386e-9891-473c-83ad-ccc05365c3ff None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:13.805056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e386e-9891-473c-83ad-ccc05365c3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:13.828330 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-6f8e386e-9891-473c-83ad-ccc05365c3ff None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:13.829862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-6f8e386e-9891-473c-83ad-ccc05365c3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:13.858027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:13.911444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 71/143] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:14.459501 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622e24b-93a9-4959-a63d-a988bca8a718 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:14.515983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:14.590773 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622e24b-93a9-4959-a63d-a988bca8a718 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:14.635350 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edad315-81c3-4bb4-a564-b4d794c7548b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:14.636368 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edad315-81c3-4bb4-a564-b4d794c7548b None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:14.636848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edad315-81c3-4bb4-a564-b4d794c7548b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:14.637317 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edad315-81c3-4bb4-a564-b4d794c7548b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:14.655413 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edad315-81c3-4bb4-a564-b4d794c7548b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:14.661041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-4edad315-81c3-4bb4-a564-b4d794c7548b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:14.661041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:14.661041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:14.661041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:14.661041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:14.661041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:14.661041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:14.661041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:14.661041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:14.661041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:14.661041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:14.661041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:14.661041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:14.661041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:14.661041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:14.661041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:14.662045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:14.662045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:14.662045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:14.662045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:14.662045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:14.662045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:14.662045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:14.662045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:14.662045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:14.662045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:14.662045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:14.662045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:14.662045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:14.662045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:14.662045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:14.662045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:14.663478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:14.663478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:14.663478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:14.663478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:14.663478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:14.663478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:14.663478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:14.663478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:14.663478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:14.663478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:14.663478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:14.663478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:14.666273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 73/144] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:14.671968 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e386e-9891-473c-83ad-ccc05365c3ff None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:14.789151 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e386e-9891-473c-83ad-ccc05365c3ff None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:14.810097 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4466902-37df-4f22-a351-5c7c2564b449 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:14.810727 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4466902-37df-4f22-a351-5c7c2564b449 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:14.811197 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4466902-37df-4f22-a351-5c7c2564b449 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:14.812959 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 74/145] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:14.820081 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62a2cf-c651-44a8-94eb-5a2d3e24a45e None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:14.820607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62a2cf-c651-44a8-94eb-5a2d3e24a45e None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:14.821073 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62a2cf-c651-44a8-94eb-5a2d3e24a45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:14.825745 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71998a1-a710-4501-8c94-44c3d8a1d996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:14.826381 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71998a1-a710-4501-8c94-44c3d8a1d996 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:14.826659 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71998a1-a710-4501-8c94-44c3d8a1d996 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:14.826932 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71998a1-a710-4501-8c94-44c3d8a1d996 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:14.827596 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71998a1-a710-4501-8c94-44c3d8a1d996 None admin] RBAC: Adding query filter params (name=member) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:14.827894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71998a1-a710-4501-8c94-44c3d8a1d996 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:14.829480 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71998a1-a710-4501-8c94-44c3d8a1d996 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:14.838311 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 72/146] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:14.844012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71998a1-a710-4501-8c94-44c3d8a1d996 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:14.846163 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-5d62a2cf-c651-44a8-94eb-5a2d3e24a45e None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:14.847576 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-5d62a2cf-c651-44a8-94eb-5a2d3e24a45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:14.867610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:14.951828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71998a1-a710-4501-8c94-44c3d8a1d996 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:14.988338 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da1e86-1281-4548-a690-ad07b1538023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:14.989396 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da1e86-1281-4548-a690-ad07b1538023 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:14.989960 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da1e86-1281-4548-a690-ad07b1538023 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:14.990415 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da1e86-1281-4548-a690-ad07b1538023 None admin] PATH_INFO: `/v3/users/77fa2fe74be84241a0d5150ae86101fb` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:14.992339 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da1e86-1281-4548-a690-ad07b1538023 None admin] RBAC: Authorizing `identity:get_user(user_id=77fa2fe74be84241a0d5150ae86101fb)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:14.994130 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da1e86-1281-4548-a690-ad07b1538023 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:14.996306 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 73/147] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:14 2022] GET /identity/v3/users/77fa2fe74be84241a0d5150ae86101fb => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:14.998469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 75/148] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:53:15.004390 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da1e86-1281-4548-a690-ad07b1538023 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:15.117018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:15.127947 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da1e86-1281-4548-a690-ad07b1538023 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:15.170293 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042176d-844c-4075-9647-62d18d5096d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:15.170909 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042176d-844c-4075-9647-62d18d5096d1 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:15.171190 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042176d-844c-4075-9647-62d18d5096d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:15.171463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042176d-844c-4075-9647-62d18d5096d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:15.250704 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042176d-844c-4075-9647-62d18d5096d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:15.252179 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042176d-844c-4075-9647-62d18d5096d1 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:15.348212 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:15.361570 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 74/149] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:15.367485 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62a2cf-c651-44a8-94eb-5a2d3e24a45e None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:15.423177 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:15.469941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f868749-6cd6-4550-b121-c2c5be52096e None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:15.470277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f868749-6cd6-4550-b121-c2c5be52096e None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:15.470511 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f868749-6cd6-4550-b121-c2c5be52096e None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:15.471941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 75/150] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:15.477843 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78779ee9-e2f9-4e98-83fa-e8d9dfcd6389 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:15.478168 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78779ee9-e2f9-4e98-83fa-e8d9dfcd6389 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:15.478437 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78779ee9-e2f9-4e98-83fa-e8d9dfcd6389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:15.483251 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62a2cf-c651-44a8-94eb-5a2d3e24a45e None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:15.521876 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee4453-6dd2-4d2d-a1d8-1c92d311c2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:15.522479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-78779ee9-e2f9-4e98-83fa-e8d9dfcd6389 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:15.522594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee4453-6dd2-4d2d-a1d8-1c92d311c2f9 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:15.522821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee4453-6dd2-4d2d-a1d8-1c92d311c2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:15.523198 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee4453-6dd2-4d2d-a1d8-1c92d311c2f9 None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:15.523669 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-78779ee9-e2f9-4e98-83fa-e8d9dfcd6389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:15.524381 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee4453-6dd2-4d2d-a1d8-1c92d311c2f9 None admin] RBAC: Authorizing `identity:get_project(project_id=f8542c9d54464124a730d0ce2363e406)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:15.526105 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee4453-6dd2-4d2d-a1d8-1c92d311c2f9 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:15.527782 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 76/151] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:15 2022] GET /identity/v3/projects/f8542c9d54464124a730d0ce2363e406 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:15.537462 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee4453-6dd2-4d2d-a1d8-1c92d311c2f9 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:15.671129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:15.677347 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee4453-6dd2-4d2d-a1d8-1c92d311c2f9 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:15.722861 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0463c4-65ad-41c0-bd70-c483093128be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:15.725156 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0463c4-65ad-41c0-bd70-c483093128be None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:15.725648 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0463c4-65ad-41c0-bd70-c483093128be None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:15.726129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0463c4-65ad-41c0-bd70-c483093128be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:15.727096 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0463c4-65ad-41c0-bd70-c483093128be None admin] RBAC: Adding query filter params (role.id=934ee0e8483c496cb8813eccbe0ebf55, scope.project.id=f8542c9d54464124a730d0ce2363e406, user.id=77fa2fe74be84241a0d5150ae86101fb) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:15.727591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0463c4-65ad-41c0-bd70-c483093128be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:15.729190 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0463c4-65ad-41c0-bd70-c483093128be None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:15.736249 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 77/152] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:53:15 2022] GET /identity/v3/role_assignments?user.id=77fa2fe74be84241a0d5150ae86101fb&scope.project.id=f8542c9d54464124a730d0ce2363e406&role.id=934ee0e8483c496cb8813eccbe0ebf55 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:15.743787 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6d395-cd00-4837-a2d6-76754f81a579 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:15.744301 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6d395-cd00-4837-a2d6-76754f81a579 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:15.744724 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6d395-cd00-4837-a2d6-76754f81a579 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:15.746494 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 78/153] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:15.752780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f96769-f0d7-449f-8e94-0a58c02ba766 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:15.753287 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f96769-f0d7-449f-8e94-0a58c02ba766 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:15.753712 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f96769-f0d7-449f-8e94-0a58c02ba766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:15.775063 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-73f96769-f0d7-449f-8e94-0a58c02ba766 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:15.776639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-73f96769-f0d7-449f-8e94-0a58c02ba766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:15.811592 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 76/154] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:15.812271 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78779ee9-e2f9-4e98-83fa-e8d9dfcd6389 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:15.853708 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 79/155] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:53:16.003469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78779ee9-e2f9-4e98-83fa-e8d9dfcd6389 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:16.024890 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:16.050376 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2941fa3-a3d5-4d12-a7e9-f09745613128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:16.050955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2941fa3-a3d5-4d12-a7e9-f09745613128 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:16.051233 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2941fa3-a3d5-4d12-a7e9-f09745613128 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:16.051504 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2941fa3-a3d5-4d12-a7e9-f09745613128 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:16.063937 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2941fa3-a3d5-4d12-a7e9-f09745613128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:16.069707 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-f2941fa3-a3d5-4d12-a7e9-f09745613128 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:53:16.069707 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:16.069707 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:16.069707 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:16.069707 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:16.069707 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:16.069707 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:16.069707 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:16.069707 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:16.069707 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:16.069707 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:16.069707 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:16.069707 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:16.069707 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:16.069707 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:16.069707 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:16.070608 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:16.070608 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:16.070608 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:16.070608 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:16.070608 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:16.070608 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:16.070608 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:16.070608 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:16.070608 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:16.070608 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:16.070608 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:16.070608 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:16.070608 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:16.070608 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:16.070608 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:16.070608 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:16.071767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:16.071767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:16.071767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:16.071767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:16.071767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:16.071767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:16.071767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:16.071767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:16.071767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:16.071767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:16.071767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:53:16.071767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:16.071767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 77/156] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:16.077601 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f96769-f0d7-449f-8e94-0a58c02ba766 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:16.149006 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2941fa3-a3d5-4d12-a7e9-f09745613128 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:16.193102 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:16.203172 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f96769-f0d7-449f-8e94-0a58c02ba766 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:16.243108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53087e8f-7e1f-42ec-ae52-079adfb1cf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:16.244429 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53087e8f-7e1f-42ec-ae52-079adfb1cf32 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:16.245190 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53087e8f-7e1f-42ec-ae52-079adfb1cf32 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:16.245740 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53087e8f-7e1f-42ec-ae52-079adfb1cf32 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:16.246815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53087e8f-7e1f-42ec-ae52-079adfb1cf32 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:16.247327 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53087e8f-7e1f-42ec-ae52-079adfb1cf32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:16.249028 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53087e8f-7e1f-42ec-ae52-079adfb1cf32 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:16.259939 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 80/157] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:16.261733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813dd3d-c12b-4f8b-9474-616e9a727a22 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:16.262295 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813dd3d-c12b-4f8b-9474-616e9a727a22 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:16.262763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813dd3d-c12b-4f8b-9474-616e9a727a22 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:16.264979 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 81/158] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:16.270177 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813dd3d-c12b-4f8b-9474-616e9a727a22 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:16.351327 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2941fa3-a3d5-4d12-a7e9-f09745613128 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:16.418760 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813dd3d-c12b-4f8b-9474-616e9a727a22 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:16.437325 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67990c18-1746-4b4e-8db2-8d59e96b6109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:16.438060 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67990c18-1746-4b4e-8db2-8d59e96b6109 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:16.438342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67990c18-1746-4b4e-8db2-8d59e96b6109 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:16.438615 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67990c18-1746-4b4e-8db2-8d59e96b6109 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:16.458998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae260f-a97b-4074-b9ef-7895f203bcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:16.460501 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae260f-a97b-4074-b9ef-7895f203bcd6 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:16.460974 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae260f-a97b-4074-b9ef-7895f203bcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:16.461587 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae260f-a97b-4074-b9ef-7895f203bcd6 None admin] PATH_INFO: `/v3/users/11f11d9b47f746229df9aeeead73a6ad` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:16.463019 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ae260f-a97b-4074-b9ef-7895f203bcd6 None admin] RBAC: Authorizing `identity:get_user(user_id=11f11d9b47f746229df9aeeead73a6ad)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:16.464668 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ae260f-a97b-4074-b9ef-7895f203bcd6 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:16.466232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67990c18-1746-4b4e-8db2-8d59e96b6109 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:16.466860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 82/159] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:16 2022] GET /identity/v3/users/11f11d9b47f746229df9aeeead73a6ad => generated 289 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:16.468417 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a1d6e-f1d2-475c-b0a4-082d3d1dde25 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:16.468918 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a1d6e-f1d2-475c-b0a4-082d3d1dde25 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:16.469402 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a1d6e-f1d2-475c-b0a4-082d3d1dde25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:16.480595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-67990c18-1746-4b4e-8db2-8d59e96b6109 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:16.480595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:16.480595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:16.480595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:16.480595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:16.480595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:16.480595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:16.480595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:16.480595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:16.480595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:16.480595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:16.480595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:16.480595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:16.480595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:16.480595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:16.480595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:16.481536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:16.481536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:16.481536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:16.481536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:16.481536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:16.481536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:16.481536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:16.481536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:16.481536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:16.481536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:16.481536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:16.481536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:16.481536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:16.481536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:16.481536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:16.481536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:16.482961 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:16.482961 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:16.482961 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:16.482961 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:16.482961 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:16.482961 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:16.482961 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:16.482961 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:16.482961 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:16.482961 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:16.482961 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:16.482961 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:16.482961 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 78/160] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:53:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:53:16.483643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67990c18-1746-4b4e-8db2-8d59e96b6109 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:16.493942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-ef1a1d6e-f1d2-475c-b0a4-082d3d1dde25 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:16.496155 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-ef1a1d6e-f1d2-475c-b0a4-082d3d1dde25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:16.517916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:16.531320 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:16.587116 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 83/161] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:16.587794 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a1d6e-f1d2-475c-b0a4-082d3d1dde25 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:16.607796 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67990c18-1746-4b4e-8db2-8d59e96b6109 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:16.650450 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded57352-773a-4279-810a-3e35e76eb4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:16.651093 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded57352-773a-4279-810a-3e35e76eb4bf None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:16.651373 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded57352-773a-4279-810a-3e35e76eb4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:16.651647 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded57352-773a-4279-810a-3e35e76eb4bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:16.664257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded57352-773a-4279-810a-3e35e76eb4bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:16.669388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-ded57352-773a-4279-810a-3e35e76eb4bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:16.669388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:16.669388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:16.669388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:16.669388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:16.669388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:16.669388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:16.669388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:16.669388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:16.669388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:16.669388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:16.669388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:16.669388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:16.669388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:16.669388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:16.669388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:16.670308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:16.670308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:16.670308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:16.670308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:16.670308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:16.670308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:16.670308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:16.670308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:16.670308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:16.670308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:16.670308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:16.670308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:16.670308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:16.670308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:16.670308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:16.670308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:16.671572 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:16.671572 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:16.671572 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:16.671572 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:16.671572 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:16.671572 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:16.671572 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:16.671572 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:16.671572 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:16.671572 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:16.671572 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:16.671572 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:16.671572 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 79/162] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:16.672447 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded57352-773a-4279-810a-3e35e76eb4bf None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:16.713146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a1d6e-f1d2-475c-b0a4-082d3d1dde25 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:16.755544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbc036-db3d-401a-bb21-8e72778fa974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:16.757066 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbc036-db3d-401a-bb21-8e72778fa974 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:16.757557 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbc036-db3d-401a-bb21-8e72778fa974 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:16.758318 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbc036-db3d-401a-bb21-8e72778fa974 None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:16.759763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecbc036-db3d-401a-bb21-8e72778fa974 None admin] RBAC: Authorizing `identity:get_project(project_id=f8542c9d54464124a730d0ce2363e406)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:16.762603 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecbc036-db3d-401a-bb21-8e72778fa974 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:16.765839 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 84/163] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:16 2022] GET /identity/v3/projects/f8542c9d54464124a730d0ce2363e406 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:16.767582 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbc036-db3d-401a-bb21-8e72778fa974 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:16.793417 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:16.802672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:16.813920 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded57352-773a-4279-810a-3e35e76eb4bf None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:16.883412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca39d3-9c21-4d1d-8803-84b460e36706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:16.884367 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca39d3-9c21-4d1d-8803-84b460e36706 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:16.884791 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca39d3-9c21-4d1d-8803-84b460e36706 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:16.885207 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca39d3-9c21-4d1d-8803-84b460e36706 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:16.886048 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca39d3-9c21-4d1d-8803-84b460e36706 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:16.886496 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca39d3-9c21-4d1d-8803-84b460e36706 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:16.887591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbc036-db3d-401a-bb21-8e72778fa974 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:16.888066 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca39d3-9c21-4d1d-8803-84b460e36706 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:16.904385 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 80/164] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:53:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:16.905298 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca39d3-9c21-4d1d-8803-84b460e36706 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:16.952849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c3be4-a41f-425f-ae5d-d154db345c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:16.953622 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c3be4-a41f-425f-ae5d-d154db345c57 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:16.954171 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c3be4-a41f-425f-ae5d-d154db345c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:16.954671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c3be4-a41f-425f-ae5d-d154db345c57 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:16.955536 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c3be4-a41f-425f-ae5d-d154db345c57 None admin] RBAC: Adding query filter params (name=member) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:16.956038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c3be4-a41f-425f-ae5d-d154db345c57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:16.957608 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c3be4-a41f-425f-ae5d-d154db345c57 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:16.975658 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 85/165] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:16.976972 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c3be4-a41f-425f-ae5d-d154db345c57 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:17.030808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca39d3-9c21-4d1d-8803-84b460e36706 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:17.070789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c7206-7019-4491-b00e-46bbd8ebd3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:17.071697 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c7206-7019-4491-b00e-46bbd8ebd3bc None admin] REQUEST_METHOD: `PUT` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:17.072121 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c7206-7019-4491-b00e-46bbd8ebd3bc None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:17.073078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c7206-7019-4491-b00e-46bbd8ebd3bc None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406/users/11f11d9b47f746229df9aeeead73a6ad/roles/e5ce565bb4ca4bad8c14b5e76d2ae8ea` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:17.075474 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c7206-7019-4491-b00e-46bbd8ebd3bc None admin] RBAC: Authorizing `identity:create_grant(project_id=f8542c9d54464124a730d0ce2363e406, user_id=11f11d9b47f746229df9aeeead73a6ad, role_id=e5ce565bb4ca4bad8c14b5e76d2ae8ea)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:17.077340 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c7206-7019-4491-b00e-46bbd8ebd3bc None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:17.095164 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 81/166] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:53:16 2022] PUT /identity/v3/projects/f8542c9d54464124a730d0ce2363e406/users/11f11d9b47f746229df9aeeead73a6ad/roles/e5ce565bb4ca4bad8c14b5e76d2ae8ea => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:53:17.096834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c7206-7019-4491-b00e-46bbd8ebd3bc None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:17.124322 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:17.136030 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:17.181679 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c3be4-a41f-425f-ae5d-d154db345c57 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:17.229549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c7206-7019-4491-b00e-46bbd8ebd3bc None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:17.253955 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f6c15-405f-44ca-8eb6-9337d78d22c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:17.255023 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f6c15-405f-44ca-8eb6-9337d78d22c3 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:17.255498 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f6c15-405f-44ca-8eb6-9337d78d22c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:17.255964 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f6c15-405f-44ca-8eb6-9337d78d22c3 None admin] PATH_INFO: `/v3/users/77fa2fe74be84241a0d5150ae86101fb` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:17.258851 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f6c15-405f-44ca-8eb6-9337d78d22c3 None admin] RBAC: Authorizing `identity:get_user(user_id=77fa2fe74be84241a0d5150ae86101fb)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:17.260755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f6c15-405f-44ca-8eb6-9337d78d22c3 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:17.263042 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 86/167] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:16 2022] GET /identity/v3/users/77fa2fe74be84241a0d5150ae86101fb => generated 275 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:17.264113 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f6c15-405f-44ca-8eb6-9337d78d22c3 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:17.273668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697f893-9692-4521-9943-d9efbe61e97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:17.275345 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697f893-9692-4521-9943-d9efbe61e97b None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:17.275628 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697f893-9692-4521-9943-d9efbe61e97b None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:17.275902 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697f893-9692-4521-9943-d9efbe61e97b None admin] PATH_INFO: `/v3/users/77fa2fe74be84241a0d5150ae86101fb` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:17.277236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8697f893-9692-4521-9943-d9efbe61e97b None admin] RBAC: Authorizing `identity:get_user(user_id=77fa2fe74be84241a0d5150ae86101fb)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:17.278785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8697f893-9692-4521-9943-d9efbe61e97b None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:17.280470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 82/168] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:17 2022] GET /identity/v3/users/77fa2fe74be84241a0d5150ae86101fb => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:17.284531 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697f893-9692-4521-9943-d9efbe61e97b None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:17.396138 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:17.412695 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697f893-9692-4521-9943-d9efbe61e97b None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:17.452030 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f6c15-405f-44ca-8eb6-9337d78d22c3 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:17.456837 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7edf7-0c51-411b-adb7-aff09827b821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:17.457461 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7edf7-0c51-411b-adb7-aff09827b821 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:17.457741 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7edf7-0c51-411b-adb7-aff09827b821 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:17.458030 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7edf7-0c51-411b-adb7-aff09827b821 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:17.491944 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2850d-b4e6-41fb-a6ab-61821250e405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:17.493088 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2850d-b4e6-41fb-a6ab-61821250e405 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:17.493760 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2850d-b4e6-41fb-a6ab-61821250e405 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:17.494322 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2850d-b4e6-41fb-a6ab-61821250e405 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:17.507265 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2850d-b4e6-41fb-a6ab-61821250e405 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:17.513673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-fca2850d-b4e6-41fb-a6ab-61821250e405 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:53:17.513673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:17.513673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:17.513673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:17.513673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:17.513673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:17.513673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:17.513673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:17.513673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:17.513673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:17.513673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:17.513673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:17.513673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:17.513673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:17.513673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:17.513673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:17.514624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:17.514624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:17.514624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:17.514624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:17.514624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:17.514624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:17.514624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:17.514624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:17.514624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:17.514624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:17.514624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:17.514624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:17.514624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:17.514624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:17.514624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:17.514624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:17.515411 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:17.515411 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:17.515411 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:17.515411 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:17.515411 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:17.515411 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:17.515411 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:17.515411 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:17.515411 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:17.515411 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:17.515411 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:53:17.515411 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:17.517891 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 87/169] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:53:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:17.519072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2850d-b4e6-41fb-a6ab-61821250e405 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:17.532843 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:17.546931 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7edf7-0c51-411b-adb7-aff09827b821 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:17.548569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7edf7-0c51-411b-adb7-aff09827b821 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:17.653078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:17.668697 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 83/170] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:17.669685 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7edf7-0c51-411b-adb7-aff09827b821 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:17.678288 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2850d-b4e6-41fb-a6ab-61821250e405 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:17.736878 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d8446-6475-4ab7-852f-d03d2bc3668f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:17.738333 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d8446-6475-4ab7-852f-d03d2bc3668f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:17.738968 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d8446-6475-4ab7-852f-d03d2bc3668f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:17.739492 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d8446-6475-4ab7-852f-d03d2bc3668f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:17.782085 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7edf7-0c51-411b-adb7-aff09827b821 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:17.823429 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a26ed-d551-4ae1-81b1-406529192c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:17.824055 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a26ed-d551-4ae1-81b1-406529192c39 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:17.824305 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a26ed-d551-4ae1-81b1-406529192c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:17.824570 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a26ed-d551-4ae1-81b1-406529192c39 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:17.825236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a26ed-d551-4ae1-81b1-406529192c39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:17.825531 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a26ed-d551-4ae1-81b1-406529192c39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:17.827409 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a26ed-d551-4ae1-81b1-406529192c39 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:17.838260 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 84/171] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:17.839730 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a26ed-d551-4ae1-81b1-406529192c39 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:17.841747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9d8446-6475-4ab7-852f-d03d2bc3668f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:17.843801 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9d8446-6475-4ab7-852f-d03d2bc3668f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:17.855994 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:17.901607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:17.963762 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 88/172] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:17.964987 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d8446-6475-4ab7-852f-d03d2bc3668f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:17.969297 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a26ed-d551-4ae1-81b1-406529192c39 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:18.014892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b377c39-bfc2-481b-a7db-21600fbe5293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:18.015540 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b377c39-bfc2-481b-a7db-21600fbe5293 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:18.015822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b377c39-bfc2-481b-a7db-21600fbe5293 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:18.016099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b377c39-bfc2-481b-a7db-21600fbe5293 None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:18.017491 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b377c39-bfc2-481b-a7db-21600fbe5293 None admin] RBAC: Authorizing `identity:get_project(project_id=f8542c9d54464124a730d0ce2363e406)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:18.019697 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b377c39-bfc2-481b-a7db-21600fbe5293 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:18.021615 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 85/173] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:17 2022] GET /identity/v3/projects/f8542c9d54464124a730d0ce2363e406 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:18.023439 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b377c39-bfc2-481b-a7db-21600fbe5293 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:18.139046 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b377c39-bfc2-481b-a7db-21600fbe5293 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:18.151927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d8446-6475-4ab7-852f-d03d2bc3668f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:18.159438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:18.169766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:18.180398 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202a641-fcee-4cbe-8e7a-865ae1f7d446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:18.181046 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202a641-fcee-4cbe-8e7a-865ae1f7d446 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:18.181326 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202a641-fcee-4cbe-8e7a-865ae1f7d446 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:18.181595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202a641-fcee-4cbe-8e7a-865ae1f7d446 None admin] PATH_INFO: `/v3/projects/2dafd3962087427ba4d7c7560d5fa88c` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:18.182743 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202a641-fcee-4cbe-8e7a-865ae1f7d446 None admin] RBAC: Authorizing `identity:get_project(project_id=2dafd3962087427ba4d7c7560d5fa88c)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:18.184748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202a641-fcee-4cbe-8e7a-865ae1f7d446 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:18.186423 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 86/174] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:18 2022] GET /identity/v3/projects/2dafd3962087427ba4d7c7560d5fa88c => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:18.187545 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202a641-fcee-4cbe-8e7a-865ae1f7d446 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:18.250768 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfa6fe-b3df-43d6-80d8-9adf9cfcbfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:18.251346 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfa6fe-b3df-43d6-80d8-9adf9cfcbfb2 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:18.251628 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfa6fe-b3df-43d6-80d8-9adf9cfcbfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:18.251906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfa6fe-b3df-43d6-80d8-9adf9cfcbfb2 None admin] PATH_INFO: `/v3/users/25374d78618f4e32be8780901a31679f` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:18.253077 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcfa6fe-b3df-43d6-80d8-9adf9cfcbfb2 None admin] RBAC: Authorizing `identity:get_user(user_id=25374d78618f4e32be8780901a31679f)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:18.254577 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcfa6fe-b3df-43d6-80d8-9adf9cfcbfb2 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:18.256736 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 89/175] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:17 2022] GET /identity/v3/users/25374d78618f4e32be8780901a31679f => generated 247 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:18.257498 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfa6fe-b3df-43d6-80d8-9adf9cfcbfb2 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:18.303517 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202a641-fcee-4cbe-8e7a-865ae1f7d446 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:18.351918 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce3f65-4f0a-4597-b7c1-093248979b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:18.351918 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce3f65-4f0a-4597-b7c1-093248979b21 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:18.351918 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce3f65-4f0a-4597-b7c1-093248979b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:18.351918 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce3f65-4f0a-4597-b7c1-093248979b21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:18.352342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce3f65-4f0a-4597-b7c1-093248979b21 None admin] RBAC: Adding query filter params (role.id=e842ac50986946a4971a51a06a39e16b, scope.project.id=f8542c9d54464124a730d0ce2363e406, user.id=77fa2fe74be84241a0d5150ae86101fb) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:18.352342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce3f65-4f0a-4597-b7c1-093248979b21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:18.353940 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce3f65-4f0a-4597-b7c1-093248979b21 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:18.360936 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 87/176] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:53:18 2022] GET /identity/v3/role_assignments?user.id=77fa2fe74be84241a0d5150ae86101fb&scope.project.id=f8542c9d54464124a730d0ce2363e406&role.id=e842ac50986946a4971a51a06a39e16b => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:18.361862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce3f65-4f0a-4597-b7c1-093248979b21 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:18.423510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:18.476251 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfa6fe-b3df-43d6-80d8-9adf9cfcbfb2 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:18.484336 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce3f65-4f0a-4597-b7c1-093248979b21 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:18.525190 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874cc3f-1ae8-4b3f-9aa0-6099be37f3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:18.525915 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874cc3f-1ae8-4b3f-9aa0-6099be37f3b8 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:18.526305 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874cc3f-1ae8-4b3f-9aa0-6099be37f3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:18.526675 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874cc3f-1ae8-4b3f-9aa0-6099be37f3b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:18.548846 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0be49-48e0-4ab4-b510-f293069a8a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:18.550206 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0be49-48e0-4ab4-b510-f293069a8a95 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:18.550778 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0be49-48e0-4ab4-b510-f293069a8a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:18.551200 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0be49-48e0-4ab4-b510-f293069a8a95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:18.552269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0be49-48e0-4ab4-b510-f293069a8a95 None admin] RBAC: Adding query filter params (role.id=934ee0e8483c496cb8813eccbe0ebf55, scope.project.id=2dafd3962087427ba4d7c7560d5fa88c, user.id=77fa2fe74be84241a0d5150ae86101fb) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:18.552735 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0be49-48e0-4ab4-b510-f293069a8a95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:18.554528 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0be49-48e0-4ab4-b510-f293069a8a95 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:18.568175 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 90/177] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:53:18 2022] GET /identity/v3/role_assignments?user.id=77fa2fe74be84241a0d5150ae86101fb&scope.project.id=2dafd3962087427ba4d7c7560d5fa88c&role.id=934ee0e8483c496cb8813eccbe0ebf55 => generated 608 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:18.570459 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ac2be-fcd5-4c32-93ab-2596f3309c5f None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:18.570984 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ac2be-fcd5-4c32-93ab-2596f3309c5f None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:18.571457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ac2be-fcd5-4c32-93ab-2596f3309c5f None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:18.573052 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 91/178] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:18.578893 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3bcbc3-c1c0-40e5-8487-3e87bf66ac83 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:18.579539 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3bcbc3-c1c0-40e5-8487-3e87bf66ac83 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:18.580020 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3bcbc3-c1c0-40e5-8487-3e87bf66ac83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:18.610022 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874cc3f-1ae8-4b3f-9aa0-6099be37f3b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:18.611347 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-0b3bcbc3-c1c0-40e5-8487-3e87bf66ac83 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:18.611489 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874cc3f-1ae8-4b3f-9aa0-6099be37f3b8 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:18.612977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-0b3bcbc3-c1c0-40e5-8487-3e87bf66ac83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:18.682426 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:18.708226 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:18.749397 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 88/179] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:18.752516 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874cc3f-1ae8-4b3f-9aa0-6099be37f3b8 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:18.818848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 92/180] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:18.869864 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874cc3f-1ae8-4b3f-9aa0-6099be37f3b8 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:18.906539 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86069eeb-4a07-49f4-8601-72d6cbedb9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:18.907192 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86069eeb-4a07-49f4-8601-72d6cbedb9fb None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:18.907468 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86069eeb-4a07-49f4-8601-72d6cbedb9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:18.907735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86069eeb-4a07-49f4-8601-72d6cbedb9fb None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:18.908985 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86069eeb-4a07-49f4-8601-72d6cbedb9fb None admin] RBAC: Authorizing `identity:get_project(project_id=f8542c9d54464124a730d0ce2363e406)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:18.910569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86069eeb-4a07-49f4-8601-72d6cbedb9fb None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:18.912212 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 89/181] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:18 2022] GET /identity/v3/projects/f8542c9d54464124a730d0ce2363e406 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:18.918413 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3bcbc3-c1c0-40e5-8487-3e87bf66ac83 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:18.994969 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9453fdd-9288-496f-845f-83f098adcedc None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:18.995341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9453fdd-9288-496f-845f-83f098adcedc None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:18.995728 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9453fdd-9288-496f-845f-83f098adcedc None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:18.997474 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 90/182] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:53:19.004955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ba9fc-6df1-4d80-9a50-f620b1a55471 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:19.005235 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ba9fc-6df1-4d80-9a50-f620b1a55471 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:19.005506 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ba9fc-6df1-4d80-9a50-f620b1a55471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:19.028673 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-1f7ba9fc-6df1-4d80-9a50-f620b1a55471 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:19.031574 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-1f7ba9fc-6df1-4d80-9a50-f620b1a55471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:19.056980 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:19.077725 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3bcbc3-c1c0-40e5-8487-3e87bf66ac83 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:19.114152 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 91/183] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:53:19.143917 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd7b9d2-a5fa-462a-a3f6-64e3bba17f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:19.144554 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7b9d2-a5fa-462a-a3f6-64e3bba17f5a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:19.144836 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7b9d2-a5fa-462a-a3f6-64e3bba17f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:19.145110 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7b9d2-a5fa-462a-a3f6-64e3bba17f5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:19.145859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd7b9d2-a5fa-462a-a3f6-64e3bba17f5a None admin] RBAC: Adding query filter params (role.id=aa420c2fb1ab4bb3b05b004a4d89779a, scope.project.id=f8542c9d54464124a730d0ce2363e406, user.id=25374d78618f4e32be8780901a31679f) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:19.146193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd7b9d2-a5fa-462a-a3f6-64e3bba17f5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:19.147732 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd7b9d2-a5fa-462a-a3f6-64e3bba17f5a None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:19.159186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 93/184] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:53:18 2022] GET /identity/v3/role_assignments?user.id=25374d78618f4e32be8780901a31679f&scope.project.id=f8542c9d54464124a730d0ce2363e406&role.id=aa420c2fb1ab4bb3b05b004a4d89779a => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:19.250179 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ba9fc-6df1-4d80-9a50-f620b1a55471 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:19.280763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90095030-4747-44cf-9caa-2199b2f7af82 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:19.281045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90095030-4747-44cf-9caa-2199b2f7af82 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:19.281326 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90095030-4747-44cf-9caa-2199b2f7af82 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:19.282857 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 94/185] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:53:19.289459 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ceca6-1f80-4c39-b026-9198390d694d None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:19.289745 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ceca6-1f80-4c39-b026-9198390d694d None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:19.290060 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ceca6-1f80-4c39-b026-9198390d694d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:19.308771 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-0b9ceca6-1f80-4c39-b026-9198390d694d None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:19.310046 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-0b9ceca6-1f80-4c39-b026-9198390d694d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:19.342895 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:19.398009 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ba9fc-6df1-4d80-9a50-f620b1a55471 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:19.431682 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 95/186] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:53:19.434370 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f56ac3-f605-4655-8eba-6ee592a3426d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:19.435013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f56ac3-f605-4655-8eba-6ee592a3426d None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:19.435287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f56ac3-f605-4655-8eba-6ee592a3426d None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:19.435555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f56ac3-f605-4655-8eba-6ee592a3426d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:19.446401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f56ac3-f605-4655-8eba-6ee592a3426d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:19.451471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-49f56ac3-f605-4655-8eba-6ee592a3426d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:53:19.451471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:19.451471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:19.451471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:19.451471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:19.451471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:19.451471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:19.451471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:19.451471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:19.451471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:19.451471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:19.451471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:19.451471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:19.451471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:19.451471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:19.451471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:19.452581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:19.452581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:19.452581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:19.452581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:19.452581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:19.452581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:19.452581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:19.452581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:19.452581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:19.452581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:19.452581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:19.452581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:19.452581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:19.452581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:19.452581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:19.452581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:19.453582 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:19.453582 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:19.453582 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:19.453582 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:19.453582 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:19.453582 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:19.453582 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:19.453582 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:19.453582 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:19.453582 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:19.453582 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:53:19.453582 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:19.453582 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 92/187] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:53:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:19.458513 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ceca6-1f80-4c39-b026-9198390d694d None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:19.464019 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f56ac3-f605-4655-8eba-6ee592a3426d None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:19.486904 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:19.603587 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ceca6-1f80-4c39-b026-9198390d694d None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:19.620391 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f56ac3-f605-4655-8eba-6ee592a3426d None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:19.647168 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c556f-95f0-42c0-a10c-682dbd69f77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:19.648125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c556f-95f0-42c0-a10c-682dbd69f77f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:19.648609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c556f-95f0-42c0-a10c-682dbd69f77f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:19.649103 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c556f-95f0-42c0-a10c-682dbd69f77f None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:19.649977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c556f-95f0-42c0-a10c-682dbd69f77f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:19.650533 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c556f-95f0-42c0-a10c-682dbd69f77f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:19.652135 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c556f-95f0-42c0-a10c-682dbd69f77f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:19.661113 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 96/188] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:19.666914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c556f-95f0-42c0-a10c-682dbd69f77f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:19.678824 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba397cd1-9f9f-495f-88ac-652a9671c154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:19.679481 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba397cd1-9f9f-495f-88ac-652a9671c154 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:19.679758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba397cd1-9f9f-495f-88ac-652a9671c154 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:19.680096 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba397cd1-9f9f-495f-88ac-652a9671c154 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:19.692179 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba397cd1-9f9f-495f-88ac-652a9671c154 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:19.698811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-ba397cd1-9f9f-495f-88ac-652a9671c154 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:19.698811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:19.698811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:19.698811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:19.698811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:19.698811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:19.698811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:19.698811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:19.698811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:19.698811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:19.698811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:19.698811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:19.698811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:19.698811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:19.698811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:19.698811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:19.699735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:19.699735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:19.699735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:19.699735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:19.699735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:19.699735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:19.699735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:19.699735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:19.699735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:19.699735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:19.699735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:19.699735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:19.699735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:19.699735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:19.699735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:19.699735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:19.700577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:19.700577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:19.700577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:19.700577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:19.700577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:19.700577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:19.700577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:19.700577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:19.700577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:19.700577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:19.700577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:19.700577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:19.705228 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 93/189] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:53:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:19.706262 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba397cd1-9f9f-495f-88ac-652a9671c154 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:19.707384 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:19.791286 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:19.810151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c556f-95f0-42c0-a10c-682dbd69f77f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:19.839799 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba397cd1-9f9f-495f-88ac-652a9671c154 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:19.848250 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed61209-9a8b-4ee0-96a4-9baacb2b2945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:19.848954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed61209-9a8b-4ee0-96a4-9baacb2b2945 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:19.849173 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed61209-9a8b-4ee0-96a4-9baacb2b2945 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:19.849440 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed61209-9a8b-4ee0-96a4-9baacb2b2945 None admin] PATH_INFO: `/v3/users/4b68f96c742f4306b09d7c0c949ad65a` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:19.863927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed61209-9a8b-4ee0-96a4-9baacb2b2945 None admin] RBAC: Authorizing `identity:get_user(user_id=4b68f96c742f4306b09d7c0c949ad65a)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:19.865403 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed61209-9a8b-4ee0-96a4-9baacb2b2945 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:19.872031 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 97/190] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:19 2022] GET /identity/v3/users/4b68f96c742f4306b09d7c0c949ad65a => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:19.872031 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed61209-9a8b-4ee0-96a4-9baacb2b2945 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:19.885795 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb8cbf-ce5a-40fc-9780-ecdb50821a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:19.886462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb8cbf-ce5a-40fc-9780-ecdb50821a34 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:19.886717 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb8cbf-ce5a-40fc-9780-ecdb50821a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:19.886982 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb8cbf-ce5a-40fc-9780-ecdb50821a34 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:19.887646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb8cbf-ce5a-40fc-9780-ecdb50821a34 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:19.887941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb8cbf-ce5a-40fc-9780-ecdb50821a34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:19.889323 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb8cbf-ce5a-40fc-9780-ecdb50821a34 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:19.898553 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 94/191] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:53:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:19.899253 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb8cbf-ce5a-40fc-9780-ecdb50821a34 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:19.972680 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:19.990038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed61209-9a8b-4ee0-96a4-9baacb2b2945 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:20.018355 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb8cbf-ce5a-40fc-9780-ecdb50821a34 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:20.029599 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d6256e-a1d8-45ed-a3f9-1985394079c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:20.030261 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6256e-a1d8-45ed-a3f9-1985394079c2 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:20.030535 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6256e-a1d8-45ed-a3f9-1985394079c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:20.030803 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6256e-a1d8-45ed-a3f9-1985394079c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:20.031896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d6256e-a1d8-45ed-a3f9-1985394079c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:20.033335 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d6256e-a1d8-45ed-a3f9-1985394079c2 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:20.034932 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 98/192] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:53:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:20.041485 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d6256e-a1d8-45ed-a3f9-1985394079c2 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:20.050381 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:20.057221 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695b996-89e5-4013-8319-b5bd08a5d10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:20.057838 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695b996-89e5-4013-8319-b5bd08a5d10e None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:20.058157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695b996-89e5-4013-8319-b5bd08a5d10e None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:20.058428 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695b996-89e5-4013-8319-b5bd08a5d10e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:20.133264 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695b996-89e5-4013-8319-b5bd08a5d10e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:20.135329 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695b996-89e5-4013-8319-b5bd08a5d10e None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:20.152547 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d6256e-a1d8-45ed-a3f9-1985394079c2 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:20.193451 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4567ec-255a-46ae-a2b5-92be27c71761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:20.194145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4567ec-255a-46ae-a2b5-92be27c71761 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:20.194421 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4567ec-255a-46ae-a2b5-92be27c71761 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:20.194806 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4567ec-255a-46ae-a2b5-92be27c71761 None admin] PATH_INFO: `/v3/users/4b68f96c742f4306b09d7c0c949ad65a` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:20.195919 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4567ec-255a-46ae-a2b5-92be27c71761 None admin] RBAC: Authorizing `identity:get_user(user_id=4b68f96c742f4306b09d7c0c949ad65a)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:20.197361 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4567ec-255a-46ae-a2b5-92be27c71761 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:20.203771 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 99/193] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:20 2022] GET /identity/v3/users/4b68f96c742f4306b09d7c0c949ad65a => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:20.203771 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4567ec-255a-46ae-a2b5-92be27c71761 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:20.225729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:20.259826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 95/194] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:20.259826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695b996-89e5-4013-8319-b5bd08a5d10e None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:20.296812 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:20.318513 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4567ec-255a-46ae-a2b5-92be27c71761 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:20.358270 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148f0bf-9772-483d-9b6e-95c2352b1e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:20.358882 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148f0bf-9772-483d-9b6e-95c2352b1e2e None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:20.359156 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148f0bf-9772-483d-9b6e-95c2352b1e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:20.359421 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148f0bf-9772-483d-9b6e-95c2352b1e2e None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:20.360069 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148f0bf-9772-483d-9b6e-95c2352b1e2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:20.361516 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148f0bf-9772-483d-9b6e-95c2352b1e2e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:20.373321 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695b996-89e5-4013-8319-b5bd08a5d10e None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:20.383852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 100/195] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 14 01:53:20 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:53:20.384733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148f0bf-9772-483d-9b6e-95c2352b1e2e None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:20.420974 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33293c-5767-40d5-b36b-b56a337e9f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:20.421704 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33293c-5767-40d5-b36b-b56a337e9f2f None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:20.422029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33293c-5767-40d5-b36b-b56a337e9f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:20.422421 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33293c-5767-40d5-b36b-b56a337e9f2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:20.484414 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:20.513635 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148f0bf-9772-483d-9b6e-95c2352b1e2e None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:20.523462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33293c-5767-40d5-b36b-b56a337e9f2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:20.525001 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33293c-5767-40d5-b36b-b56a337e9f2f None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:20.560048 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:20.560339 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c60d0d-9986-4bdd-a062-2feac1d42406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:20.561177 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c60d0d-9986-4bdd-a062-2feac1d42406 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:20.561177 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c60d0d-9986-4bdd-a062-2feac1d42406 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:20.561461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c60d0d-9986-4bdd-a062-2feac1d42406 None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:20.563477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c60d0d-9986-4bdd-a062-2feac1d42406 None admin] RBAC: Authorizing `identity:get_project(project_id=60ea1e9a961b4e3b99b9fe44ac610808)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:20.564946 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c60d0d-9986-4bdd-a062-2feac1d42406 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:20.566693 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 101/196] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:20 2022] GET /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:20.567937 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1020d-ed11-41ed-851d-5a26c81a2011 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:20.568221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1020d-ed11-41ed-851d-5a26c81a2011 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:20.569019 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1020d-ed11-41ed-851d-5a26c81a2011 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:20.570474 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 102/197] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:20.576068 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1020d-ed11-41ed-851d-5a26c81a2011 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:20.668113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 96/198] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:20.669442 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4901c-e325-4de9-af2e-1e1711e192c9 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:20.669852 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4901c-e325-4de9-af2e-1e1711e192c9 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:20.670157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4901c-e325-4de9-af2e-1e1711e192c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:20.693189 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-a5a4901c-e325-4de9-af2e-1e1711e192c9 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:20.694978 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-a5a4901c-e325-4de9-af2e-1e1711e192c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:20.708375 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1020d-ed11-41ed-851d-5a26c81a2011 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:20.759605 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f4966-d920-403f-94a0-821c6be61ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:20.760267 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f4966-d920-403f-94a0-821c6be61ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:20.760547 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f4966-d920-403f-94a0-821c6be61ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:20.760815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f4966-d920-403f-94a0-821c6be61ef8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:20.761541 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f4966-d920-403f-94a0-821c6be61ef8 None admin] RBAC: Adding query filter params (role.id=aa420c2fb1ab4bb3b05b004a4d89779a, scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808, user.id=4b68f96c742f4306b09d7c0c949ad65a) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:20.761852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f4966-d920-403f-94a0-821c6be61ef8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:20.763391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f4966-d920-403f-94a0-821c6be61ef8 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:20.770787 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 103/199] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:53:20 2022] GET /identity/v3/role_assignments?user.id=4b68f96c742f4306b09d7c0c949ad65a&scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808&role.id=aa420c2fb1ab4bb3b05b004a4d89779a => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:20.771414 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f4966-d920-403f-94a0-821c6be61ef8 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:20.779801 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 97/200] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:20.786468 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:20.899351 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f4966-d920-403f-94a0-821c6be61ef8 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:20.935690 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a71d76-4840-423f-b532-6a8c939696a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:20.936717 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a71d76-4840-423f-b532-6a8c939696a1 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:20.937203 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a71d76-4840-423f-b532-6a8c939696a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:20.937679 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a71d76-4840-423f-b532-6a8c939696a1 None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:20.939240 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a71d76-4840-423f-b532-6a8c939696a1 None admin] RBAC: Authorizing `identity:get_project(project_id=60ea1e9a961b4e3b99b9fe44ac610808)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:20.941014 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a71d76-4840-423f-b532-6a8c939696a1 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:20.942965 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 104/201] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:20 2022] GET /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:20.949447 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4901c-e325-4de9-af2e-1e1711e192c9 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:21.041604 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:21.072065 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a71d76-4840-423f-b532-6a8c939696a1 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:21.091241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4901c-e325-4de9-af2e-1e1711e192c9 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:21.157784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f93a9-04a1-4aeb-91a2-ec82f8a9ad3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:21.158430 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f93a9-04a1-4aeb-91a2-ec82f8a9ad3c None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:21.158711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f93a9-04a1-4aeb-91a2-ec82f8a9ad3c None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:21.158985 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f93a9-04a1-4aeb-91a2-ec82f8a9ad3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:21.159718 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f93a9-04a1-4aeb-91a2-ec82f8a9ad3c None admin] RBAC: Adding query filter params (role.id=e842ac50986946a4971a51a06a39e16b, scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808, user.id=4b68f96c742f4306b09d7c0c949ad65a) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:21.160028 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f93a9-04a1-4aeb-91a2-ec82f8a9ad3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:21.161558 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f93a9-04a1-4aeb-91a2-ec82f8a9ad3c None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:21.167581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 98/202] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:53:20 2022] GET /identity/v3/role_assignments?user.id=4b68f96c742f4306b09d7c0c949ad65a&scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808&role.id=e842ac50986946a4971a51a06a39e16b => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:21.205971 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:21.240100 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a71d76-4840-423f-b532-6a8c939696a1 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:21.299876 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ed692-ed43-4d94-adde-cf35e7dcda5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:21.300533 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ed692-ed43-4d94-adde-cf35e7dcda5c None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:21.303420 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ed692-ed43-4d94-adde-cf35e7dcda5c None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:21.303791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ed692-ed43-4d94-adde-cf35e7dcda5c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:21.321133 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ed692-ed43-4d94-adde-cf35e7dcda5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:21.327245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-6a0ed692-ed43-4d94-adde-cf35e7dcda5c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:53:21.327245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:21.327245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:21.327245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:21.327245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:21.327245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:21.327245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:21.327245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:21.327245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:21.327245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:21.327245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:21.327245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:21.327245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:21.327245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:21.327245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:21.327245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:21.328218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:21.328218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:21.328218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:21.328218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:21.328218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:21.328218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:21.328218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:21.328218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:21.328218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:21.328218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:21.328218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:21.328218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:21.328218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:21.328218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:21.328218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:21.328218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:21.329038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:21.329038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:21.329038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:21.329038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:21.329038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:21.329038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:21.329038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:21.329038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:21.329038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:21.329038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:21.329038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:53:21.329038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:21.331823 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 105/203] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:53:21.341927 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f93a9-04a1-4aeb-91a2-ec82f8a9ad3c None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:21.457950 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f93a9-04a1-4aeb-91a2-ec82f8a9ad3c None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:21.484987 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:21.496887 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79037541-6cda-4259-aad3-e9f0e2acbce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:21.497501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79037541-6cda-4259-aad3-e9f0e2acbce1 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:21.497780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79037541-6cda-4259-aad3-e9f0e2acbce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:21.498078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79037541-6cda-4259-aad3-e9f0e2acbce1 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:21.498792 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79037541-6cda-4259-aad3-e9f0e2acbce1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:21.499099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79037541-6cda-4259-aad3-e9f0e2acbce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:21.500471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79037541-6cda-4259-aad3-e9f0e2acbce1 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:21.510123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 99/204] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:21.515676 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ed692-ed43-4d94-adde-cf35e7dcda5c None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:21.642410 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ed692-ed43-4d94-adde-cf35e7dcda5c None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:21.694928 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede9a87-fb8d-446b-9475-cc644e78793d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:21.695941 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede9a87-fb8d-446b-9475-cc644e78793d None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:21.696424 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede9a87-fb8d-446b-9475-cc644e78793d None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:21.696898 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede9a87-fb8d-446b-9475-cc644e78793d None admin] PATH_INFO: `/v3/users/11f11d9b47f746229df9aeeead73a6ad` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:21.698336 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede9a87-fb8d-446b-9475-cc644e78793d None admin] RBAC: Authorizing `identity:get_user(user_id=11f11d9b47f746229df9aeeead73a6ad)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:21.700015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede9a87-fb8d-446b-9475-cc644e78793d None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:21.701815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 106/205] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:21 2022] GET /identity/v3/users/11f11d9b47f746229df9aeeead73a6ad => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:21.707396 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79037541-6cda-4259-aad3-e9f0e2acbce1 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:21.819266 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79037541-6cda-4259-aad3-e9f0e2acbce1 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:21.854648 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc6d2c-bc30-4f0e-b228-654d11955fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:21.855265 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc6d2c-bc30-4f0e-b228-654d11955fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:21.855544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc6d2c-bc30-4f0e-b228-654d11955fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:21.855813 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc6d2c-bc30-4f0e-b228-654d11955fe8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:21.916906 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:21.928465 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc6d2c-bc30-4f0e-b228-654d11955fe8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:21.929942 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc6d2c-bc30-4f0e-b228-654d11955fe8 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:22.040993 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 100/206] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:22.046659 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede9a87-fb8d-446b-9475-cc644e78793d None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:22.065010 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:22.185020 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede9a87-fb8d-446b-9475-cc644e78793d None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:22.220582 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f0de1-5cc8-4ac2-85f1-9fec2358c98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:22.221558 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f0de1-5cc8-4ac2-85f1-9fec2358c98b None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:22.222083 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f0de1-5cc8-4ac2-85f1-9fec2358c98b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:22.222554 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f0de1-5cc8-4ac2-85f1-9fec2358c98b None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:22.223893 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f0de1-5cc8-4ac2-85f1-9fec2358c98b None admin] RBAC: Authorizing `identity:get_project(project_id=f8542c9d54464124a730d0ce2363e406)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:22.226139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f0de1-5cc8-4ac2-85f1-9fec2358c98b None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:22.228325 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 107/207] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:22 2022] GET /identity/v3/projects/f8542c9d54464124a730d0ce2363e406 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:22.235896 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc6d2c-bc30-4f0e-b228-654d11955fe8 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:22.334774 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:22.357328 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc6d2c-bc30-4f0e-b228-654d11955fe8 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:22.412186 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2c178-f997-4d2a-9d82-016431fe2869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:22.412813 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2c178-f997-4d2a-9d82-016431fe2869 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:22.413086 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2c178-f997-4d2a-9d82-016431fe2869 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:22.413351 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2c178-f997-4d2a-9d82-016431fe2869 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:22.414179 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2c178-f997-4d2a-9d82-016431fe2869 None admin] RBAC: Adding query filter params (role.id=e5ce565bb4ca4bad8c14b5e76d2ae8ea, scope.project.id=f8542c9d54464124a730d0ce2363e406, user.id=11f11d9b47f746229df9aeeead73a6ad) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:22.414379 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2c178-f997-4d2a-9d82-016431fe2869 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:22.415824 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2c178-f997-4d2a-9d82-016431fe2869 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:22.426656 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 101/208] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:53:22 2022] GET /identity/v3/role_assignments?user.id=11f11d9b47f746229df9aeeead73a6ad&scope.project.id=f8542c9d54464124a730d0ce2363e406&role.id=e5ce565bb4ca4bad8c14b5e76d2ae8ea => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:22.833526 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db09604-31d0-4ad3-ad19-8069cdcce744 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:22.834823 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db09604-31d0-4ad3-ad19-8069cdcce744 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:22.835329 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db09604-31d0-4ad3-ad19-8069cdcce744 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:22.837099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 108/209] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:22.842851 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642bb89-014d-4a7b-ab6f-50956942aff7 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:22.843353 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642bb89-014d-4a7b-ab6f-50956942aff7 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:22.843819 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642bb89-014d-4a7b-ab6f-50956942aff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:22.870438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-d642bb89-014d-4a7b-ab6f-50956942aff7 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:22.871949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-d642bb89-014d-4a7b-ab6f-50956942aff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:22.963735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 102/210] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:53:23.253215 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db09604-31d0-4ad3-ad19-8069cdcce744 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:23.336403 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:23.371912 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db09604-31d0-4ad3-ad19-8069cdcce744 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:23.419777 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978053b-437f-49b7-909c-a621f2a87107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:23.420760 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978053b-437f-49b7-909c-a621f2a87107 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:23.421237 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978053b-437f-49b7-909c-a621f2a87107 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:23.421733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978053b-437f-49b7-909c-a621f2a87107 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:23.434306 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d978053b-437f-49b7-909c-a621f2a87107 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:23.439404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-d978053b-437f-49b7-909c-a621f2a87107 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:23.439404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:23.439404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:23.439404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:23.439404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:23.439404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:23.439404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:23.439404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:23.439404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:23.439404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:23.439404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:23.439404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:23.439404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:23.439404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:23.439404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:23.439404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:23.440294 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:23.440294 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:23.440294 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:23.440294 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:23.440294 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:23.440294 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:23.440294 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:23.440294 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:23.440294 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:23.440294 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:23.440294 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:23.440294 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:23.440294 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:23.440294 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:23.440294 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:23.440294 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:23.441098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:23.441098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:23.441098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:23.441098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:23.441098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:23.441098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:23.441098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:23.441098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:23.441098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:23.441098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:23.441098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:23.441098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:23.443785 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 109/211] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:23.449212 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642bb89-014d-4a7b-ab6f-50956942aff7 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:23.459647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fdc3a-def6-4ca5-ae77-d1b2e1cf1192 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:23.459972 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fdc3a-def6-4ca5-ae77-d1b2e1cf1192 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:23.460249 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fdc3a-def6-4ca5-ae77-d1b2e1cf1192 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:23.461735 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 110/212] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:23.466730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efaa2c-794c-4721-b425-2fb0d0162769 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:23.467021 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efaa2c-794c-4721-b425-2fb0d0162769 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:23.467315 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efaa2c-794c-4721-b425-2fb0d0162769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:23.486578 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-23efaa2c-794c-4721-b425-2fb0d0162769 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:23.487757 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-23efaa2c-794c-4721-b425-2fb0d0162769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:23.539568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:23.559071 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 111/213] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:53:23.560068 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642bb89-014d-4a7b-ab6f-50956942aff7 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:23.564314 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4eff19-99a4-492e-a3ab-4a457f0c3141 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:23.564863 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4eff19-99a4-492e-a3ab-4a457f0c3141 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:23.565384 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4eff19-99a4-492e-a3ab-4a457f0c3141 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:23.566940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 112/214] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:23.572427 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a332c1-01e0-4bb8-88b6-92e45c768cb2 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:23.572711 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a332c1-01e0-4bb8-88b6-92e45c768cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:23.572983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a332c1-01e0-4bb8-88b6-92e45c768cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:23.596963 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-31a332c1-01e0-4bb8-88b6-92e45c768cb2 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:23.598450 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-31a332c1-01e0-4bb8-88b6-92e45c768cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:23.603524 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa364dd-c338-43f0-a550-a400326014cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:23.604426 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa364dd-c338-43f0-a550-a400326014cb None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:23.604897 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa364dd-c338-43f0-a550-a400326014cb None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:23.605354 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa364dd-c338-43f0-a550-a400326014cb None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:23.606279 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa364dd-c338-43f0-a550-a400326014cb None admin] RBAC: Adding query filter params (name=member) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:23.606768 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa364dd-c338-43f0-a550-a400326014cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:23.608368 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa364dd-c338-43f0-a550-a400326014cb None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:23.617547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 103/215] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:23.623035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa364dd-c338-43f0-a550-a400326014cb None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:23.668670 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:23.684752 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 113/216] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:23.733202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa364dd-c338-43f0-a550-a400326014cb None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:23.768128 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acead2e-1f2a-490a-9af3-d946381a6415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:23.768767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acead2e-1f2a-490a-9af3-d946381a6415 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:23.769047 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acead2e-1f2a-490a-9af3-d946381a6415 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:23.769374 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acead2e-1f2a-490a-9af3-d946381a6415 None admin] PATH_INFO: `/v3/users/cd759a3de9ea4217a5287b1e8f6d0ba9` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:23.781301 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:23.786865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acead2e-1f2a-490a-9af3-d946381a6415 None admin] RBAC: Authorizing `identity:get_user(user_id=cd759a3de9ea4217a5287b1e8f6d0ba9)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:23.789481 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acead2e-1f2a-490a-9af3-d946381a6415 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:23.791809 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 104/217] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:23 2022] GET /identity/v3/users/cd759a3de9ea4217a5287b1e8f6d0ba9 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:23.796935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a332c1-01e0-4bb8-88b6-92e45c768cb2 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:23.823226 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acead2e-1f2a-490a-9af3-d946381a6415 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:23.913852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a332c1-01e0-4bb8-88b6-92e45c768cb2 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:23.950878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acead2e-1f2a-490a-9af3-d946381a6415 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:23.951313 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116fbdf0-980c-4065-b43a-dcb8715b59a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:23.952903 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fbdf0-980c-4065-b43a-dcb8715b59a0 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:23.953412 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fbdf0-980c-4065-b43a-dcb8715b59a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:23.953907 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fbdf0-980c-4065-b43a-dcb8715b59a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:23.988720 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f76bd-b306-4b2e-9121-ba0941213c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:23.989351 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f76bd-b306-4b2e-9121-ba0941213c3c None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:23.989617 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f76bd-b306-4b2e-9121-ba0941213c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:23.989881 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f76bd-b306-4b2e-9121-ba0941213c3c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:23.991610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f76bd-b306-4b2e-9121-ba0941213c3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:23.993117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f76bd-b306-4b2e-9121-ba0941213c3c None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:23.994996 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 105/218] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:53:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:23.995764 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f76bd-b306-4b2e-9121-ba0941213c3c None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:24.027793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:24.035362 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116fbdf0-980c-4065-b43a-dcb8715b59a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:24.036823 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116fbdf0-980c-4065-b43a-dcb8715b59a0 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:24.059839 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:24.091931 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f76bd-b306-4b2e-9121-ba0941213c3c None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:24.125026 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43c0e7-9c65-4f62-90df-36b3d3bc2cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:24.125653 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43c0e7-9c65-4f62-90df-36b3d3bc2cec None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:24.125932 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43c0e7-9c65-4f62-90df-36b3d3bc2cec None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:24.126231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43c0e7-9c65-4f62-90df-36b3d3bc2cec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:24.133610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 114/219] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:24.134875 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77d61c-9aa6-4f9b-9588-fa799af492dd None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:24.135159 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77d61c-9aa6-4f9b-9588-fa799af492dd None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:24.135432 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77d61c-9aa6-4f9b-9588-fa799af492dd None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:24.136779 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f43c0e7-9c65-4f62-90df-36b3d3bc2cec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:24.136885 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 115/220] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:24.137336 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77d61c-9aa6-4f9b-9588-fa799af492dd None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:24.142991 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-7f43c0e7-9c65-4f62-90df-36b3d3bc2cec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:53:24.142991 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:24.142991 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:24.142991 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:24.142991 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:24.142991 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:24.142991 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:24.142991 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:24.142991 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:24.142991 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:24.142991 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:24.142991 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:24.142991 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:24.142991 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:24.142991 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:24.142991 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:24.143880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:24.143880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:24.143880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:24.143880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:24.143880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:24.143880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:24.143880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:24.143880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:24.143880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:24.143880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:24.143880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:24.143880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:24.143880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:24.143880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:24.143880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:24.143880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:24.144718 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:24.144718 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:24.144718 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:24.144718 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:24.144718 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:24.144718 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:24.144718 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:24.144718 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:24.144718 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:24.144718 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:24.144718 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:53:24.144718 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:24.149278 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 106/221] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:53:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:24.150142 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43c0e7-9c65-4f62-90df-36b3d3bc2cec None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:24.230043 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:24.237558 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77d61c-9aa6-4f9b-9588-fa799af492dd None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:24.246111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43c0e7-9c65-4f62-90df-36b3d3bc2cec None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:24.270386 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b31708-e9c5-4b8a-989a-b321463c1746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:24.272003 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b31708-e9c5-4b8a-989a-b321463c1746 None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:24.272279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b31708-e9c5-4b8a-989a-b321463c1746 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:24.272544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b31708-e9c5-4b8a-989a-b321463c1746 None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:24.273177 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b31708-e9c5-4b8a-989a-b321463c1746 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:24.274484 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:24.274691 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b31708-e9c5-4b8a-989a-b321463c1746 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:24.282901 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c314e-9470-49b5-9ed0-2b04035fa5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:24.283533 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c314e-9470-49b5-9ed0-2b04035fa5af None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:24.283812 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c314e-9470-49b5-9ed0-2b04035fa5af None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:24.284086 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c314e-9470-49b5-9ed0-2b04035fa5af None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:24.285206 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c314e-9470-49b5-9ed0-2b04035fa5af None admin] RBAC: Authorizing `identity:get_project(project_id=60ea1e9a961b4e3b99b9fe44ac610808)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:24.287490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c314e-9470-49b5-9ed0-2b04035fa5af None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:24.294566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 107/222] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:24 2022] GET /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:24.294566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668d8dc-de09-42fb-a68f-4aa19848c73d None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:24.294566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668d8dc-de09-42fb-a68f-4aa19848c73d None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:24.294566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668d8dc-de09-42fb-a68f-4aa19848c73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:24.302568 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 116/223] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 14 01:53:24 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:53:24.303407 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b31708-e9c5-4b8a-989a-b321463c1746 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:24.316995 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-9668d8dc-de09-42fb-a68f-4aa19848c73d None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:24.318679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-9668d8dc-de09-42fb-a68f-4aa19848c73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:24.410033 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 108/224] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:24.419080 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668d8dc-de09-42fb-a68f-4aa19848c73d None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:24.438966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b31708-e9c5-4b8a-989a-b321463c1746 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:24.478239 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebd013-f461-483d-a213-3d42013fd795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:24.478885 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebd013-f461-483d-a213-3d42013fd795 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:24.479159 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebd013-f461-483d-a213-3d42013fd795 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:24.479424 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebd013-f461-483d-a213-3d42013fd795 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:24.480086 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebd013-f461-483d-a213-3d42013fd795 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:24.480730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebd013-f461-483d-a213-3d42013fd795 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:24.482680 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebd013-f461-483d-a213-3d42013fd795 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:24.492871 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 117/225] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:24.499022 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebd013-f461-483d-a213-3d42013fd795 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:24.516343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:24.557136 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668d8dc-de09-42fb-a68f-4aa19848c73d None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:24.585812 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:24.599167 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad896b2-a222-410d-882e-90ccf29b299f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:24.600261 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad896b2-a222-410d-882e-90ccf29b299f None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:24.600541 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad896b2-a222-410d-882e-90ccf29b299f None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:24.600816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad896b2-a222-410d-882e-90ccf29b299f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:24.601871 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad896b2-a222-410d-882e-90ccf29b299f None admin] RBAC: Adding query filter params (role.id=934ee0e8483c496cb8813eccbe0ebf55, scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808, user.id=cd759a3de9ea4217a5287b1e8f6d0ba9) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:24.602414 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad896b2-a222-410d-882e-90ccf29b299f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:24.603719 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad896b2-a222-410d-882e-90ccf29b299f None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:24.609602 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 109/226] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:53:24 2022] GET /identity/v3/role_assignments?user.id=cd759a3de9ea4217a5287b1e8f6d0ba9&scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808&role.id=934ee0e8483c496cb8813eccbe0ebf55 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:24.626970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebd013-f461-483d-a213-3d42013fd795 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:24.666267 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b71a4e-234b-4ca4-a3c5-37fa14c82542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:24.666927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b71a4e-234b-4ca4-a3c5-37fa14c82542 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:24.667208 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b71a4e-234b-4ca4-a3c5-37fa14c82542 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:24.667482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b71a4e-234b-4ca4-a3c5-37fa14c82542 None admin] PATH_INFO: `/v3/users/4b68f96c742f4306b09d7c0c949ad65a` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:24.668704 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b71a4e-234b-4ca4-a3c5-37fa14c82542 None admin] RBAC: Authorizing `identity:get_user(user_id=4b68f96c742f4306b09d7c0c949ad65a)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:24.670218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b71a4e-234b-4ca4-a3c5-37fa14c82542 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:24.672069 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 118/227] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:24 2022] GET /identity/v3/users/4b68f96c742f4306b09d7c0c949ad65a => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:24.678389 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad896b2-a222-410d-882e-90ccf29b299f None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:24.687463 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b71a4e-234b-4ca4-a3c5-37fa14c82542 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:24.791859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:24.819284 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b71a4e-234b-4ca4-a3c5-37fa14c82542 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:24.831817 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad896b2-a222-410d-882e-90ccf29b299f None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:24.861036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951db2e0-15a6-413d-827c-d965ad8dd000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:24.861589 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951db2e0-15a6-413d-827c-d965ad8dd000 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:24.861866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951db2e0-15a6-413d-827c-d965ad8dd000 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:24.862329 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951db2e0-15a6-413d-827c-d965ad8dd000 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:24.877268 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951db2e0-15a6-413d-827c-d965ad8dd000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:24.885873 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-951db2e0-15a6-413d-827c-d965ad8dd000 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:24.885873 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:24.885873 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:24.885873 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:24.885873 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:24.885873 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:24.885873 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:24.885873 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:24.885873 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:24.885873 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:24.885873 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:24.885873 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:24.885873 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:24.885873 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:24.885873 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:24.885873 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:24.886966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:24.886966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:24.886966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:24.886966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:24.886966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:24.886966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:24.886966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:24.886966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:24.886966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:24.886966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:24.886966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:24.886966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:24.886966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:24.886966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:24.886966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:24.886966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:24.887795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:24.887795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:24.887795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:24.887795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:24.887795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:24.887795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:24.887795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:24.887795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:24.887795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:24.887795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:24.887795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:24.887795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:24.887795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 119/228] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:53:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:53:24.894031 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951db2e0-15a6-413d-827c-d965ad8dd000 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:24.895894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc38ad-a463-4dc8-a1c5-6fb0fe51faec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:24.897072 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc38ad-a463-4dc8-a1c5-6fb0fe51faec None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:24.897514 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc38ad-a463-4dc8-a1c5-6fb0fe51faec None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:24.897792 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc38ad-a463-4dc8-a1c5-6fb0fe51faec None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:24.899867 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc38ad-a463-4dc8-a1c5-6fb0fe51faec None admin] RBAC: Authorizing `identity:get_project(project_id=60ea1e9a961b4e3b99b9fe44ac610808)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:24.901331 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc38ad-a463-4dc8-a1c5-6fb0fe51faec None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:24.903397 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 110/229] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:24 2022] GET /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:24.909786 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc38ad-a463-4dc8-a1c5-6fb0fe51faec None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:24.978505 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:25.029810 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc38ad-a463-4dc8-a1c5-6fb0fe51faec None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:25.084889 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951db2e0-15a6-413d-827c-d965ad8dd000 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:25.088276 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164e27a-c01a-4860-8c49-d9e236935a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:25.088957 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164e27a-c01a-4860-8c49-d9e236935a59 None admin] REQUEST_METHOD: `PUT` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:25.089235 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164e27a-c01a-4860-8c49-d9e236935a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:25.089509 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164e27a-c01a-4860-8c49-d9e236935a59 None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808/users/4b68f96c742f4306b09d7c0c949ad65a/roles/aa420c2fb1ab4bb3b05b004a4d89779a` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:25.092261 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164e27a-c01a-4860-8c49-d9e236935a59 None admin] RBAC: Authorizing `identity:create_grant(project_id=60ea1e9a961b4e3b99b9fe44ac610808, user_id=4b68f96c742f4306b09d7c0c949ad65a, role_id=aa420c2fb1ab4bb3b05b004a4d89779a)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:25.093859 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164e27a-c01a-4860-8c49-d9e236935a59 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:25.117408 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 111/230] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:53:24 2022] PUT /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808/users/4b68f96c742f4306b09d7c0c949ad65a/roles/aa420c2fb1ab4bb3b05b004a4d89779a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:53:25.178767 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c4767-5366-486f-89a4-619ec50c79a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:25.179417 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c4767-5366-486f-89a4-619ec50c79a7 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:25.179702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c4767-5366-486f-89a4-619ec50c79a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:25.179981 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c4767-5366-486f-89a4-619ec50c79a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:25.180681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c4767-5366-486f-89a4-619ec50c79a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:25.180991 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c4767-5366-486f-89a4-619ec50c79a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:25.182793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c4767-5366-486f-89a4-619ec50c79a7 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:25.197774 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 120/231] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:53:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:25.203509 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164e27a-c01a-4860-8c49-d9e236935a59 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:25.357312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164e27a-c01a-4860-8c49-d9e236935a59 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:25.385303 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:25.412452 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73ca52-1ea3-457b-a401-18cbda68110c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:25.413105 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73ca52-1ea3-457b-a401-18cbda68110c None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:25.413386 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73ca52-1ea3-457b-a401-18cbda68110c None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:25.413659 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73ca52-1ea3-457b-a401-18cbda68110c None admin] PATH_INFO: `/v3/users/4b68f96c742f4306b09d7c0c949ad65a` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:25.414847 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73ca52-1ea3-457b-a401-18cbda68110c None admin] RBAC: Authorizing `identity:get_user(user_id=4b68f96c742f4306b09d7c0c949ad65a)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:25.416307 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73ca52-1ea3-457b-a401-18cbda68110c None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:25.417909 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 112/232] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:25 2022] GET /identity/v3/users/4b68f96c742f4306b09d7c0c949ad65a => generated 283 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:25.423434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c4767-5366-486f-89a4-619ec50c79a7 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:25.426625 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:25.546535 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c4767-5366-486f-89a4-619ec50c79a7 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:25.582502 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edc330-dac0-461f-b407-13ca7debdaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:25.583302 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edc330-dac0-461f-b407-13ca7debdaec None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:25.583591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edc330-dac0-461f-b407-13ca7debdaec None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:25.584395 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edc330-dac0-461f-b407-13ca7debdaec None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:25.585563 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edc330-dac0-461f-b407-13ca7debdaec None admin] RBAC: Authorizing `identity:get_project(project_id=60ea1e9a961b4e3b99b9fe44ac610808)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:25.587082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edc330-dac0-461f-b407-13ca7debdaec None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:25.589083 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 121/233] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:25 2022] GET /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:25.594619 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73ca52-1ea3-457b-a401-18cbda68110c None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:25.748138 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73ca52-1ea3-457b-a401-18cbda68110c None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:25.787505 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3149cbea-2553-4a90-a649-285b7d38da59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:25.788533 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149cbea-2553-4a90-a649-285b7d38da59 None admin] REQUEST_METHOD: `PUT` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:25.788819 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149cbea-2553-4a90-a649-285b7d38da59 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:25.789137 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149cbea-2553-4a90-a649-285b7d38da59 None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808/users/4b68f96c742f4306b09d7c0c949ad65a/roles/e842ac50986946a4971a51a06a39e16b` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:25.791500 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3149cbea-2553-4a90-a649-285b7d38da59 None admin] RBAC: Authorizing `identity:create_grant(project_id=60ea1e9a961b4e3b99b9fe44ac610808, user_id=4b68f96c742f4306b09d7c0c949ad65a, role_id=e842ac50986946a4971a51a06a39e16b)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:25.792987 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3149cbea-2553-4a90-a649-285b7d38da59 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:25.809432 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 113/234] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:53:25 2022] PUT /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808/users/4b68f96c742f4306b09d7c0c949ad65a/roles/e842ac50986946a4971a51a06a39e16b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:53:27.179243 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4114302-b7df-4f68-8375-9b7f384c6a2b None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:27.179814 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4114302-b7df-4f68-8375-9b7f384c6a2b None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:27.179814 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4114302-b7df-4f68-8375-9b7f384c6a2b None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:27.181541 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 122/235] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:27.187484 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855aebf-1741-44b2-9b9e-72e76c6aefed None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:27.187992 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855aebf-1741-44b2-9b9e-72e76c6aefed None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:27.188099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855aebf-1741-44b2-9b9e-72e76c6aefed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:27.207457 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-a855aebf-1741-44b2-9b9e-72e76c6aefed None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:27.208690 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-a855aebf-1741-44b2-9b9e-72e76c6aefed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:27.257056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:27.288243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 114/236] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:27.296382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee6623-b27e-4376-a4c8-2acb51985128 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:27.296658 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee6623-b27e-4376-a4c8-2acb51985128 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:27.296936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee6623-b27e-4376-a4c8-2acb51985128 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:27.298611 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 123/237] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:27.304111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d244a1-88af-4290-8589-78eb26d0b059 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:27.304407 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d244a1-88af-4290-8589-78eb26d0b059 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:27.304679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d244a1-88af-4290-8589-78eb26d0b059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:27.323429 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-09d244a1-88af-4290-8589-78eb26d0b059 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:27.324607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-09d244a1-88af-4290-8589-78eb26d0b059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:27.401322 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 115/238] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:27.443714 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a5ba8-2f73-4f35-85ec-b6125e3852aa None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:27.444302 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a5ba8-2f73-4f35-85ec-b6125e3852aa None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:27.444776 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a5ba8-2f73-4f35-85ec-b6125e3852aa None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:27.446603 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 124/239] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:27.451950 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a44eca-3d25-42f5-b9b8-afc7b9919b70 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:27.452551 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a44eca-3d25-42f5-b9b8-afc7b9919b70 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:27.453021 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a44eca-3d25-42f5-b9b8-afc7b9919b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:27.471415 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-c2a44eca-3d25-42f5-b9b8-afc7b9919b70 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:27.472911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-c2a44eca-3d25-42f5-b9b8-afc7b9919b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:27.552203 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 116/240] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:27.569420 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a5ba8-2f73-4f35-85ec-b6125e3852aa None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:27.670083 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:27.703797 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a5ba8-2f73-4f35-85ec-b6125e3852aa None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:27.743956 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d078615-151b-44f6-936e-61a0cf449572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:27.744897 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d078615-151b-44f6-936e-61a0cf449572 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:27.745372 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d078615-151b-44f6-936e-61a0cf449572 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:27.745862 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d078615-151b-44f6-936e-61a0cf449572 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:27.766937 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d078615-151b-44f6-936e-61a0cf449572 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:27.773248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-3d078615-151b-44f6-936e-61a0cf449572 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:53:27.773248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:27.773248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:27.773248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:27.773248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:27.773248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:27.773248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:27.773248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:27.773248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:27.773248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:27.773248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:27.773248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:27.773248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:27.773248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:27.773248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:27.773248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:27.774180 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:27.774180 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:27.774180 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:27.774180 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:27.774180 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:27.774180 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:27.774180 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:27.774180 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:27.774180 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:27.774180 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:27.774180 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:27.774180 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:27.774180 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:27.774180 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:27.774180 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:27.774180 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:27.774983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:27.774983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:27.774983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:27.774983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:27.774983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:27.774983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:27.774983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:27.774983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:27.774983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:27.774983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:27.774983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:53:27.774983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:27.777480 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 125/241] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:27.783297 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a44eca-3d25-42f5-b9b8-afc7b9919b70 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:27.801366 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d078615-151b-44f6-936e-61a0cf449572 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:27.854439 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:27.924827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d078615-151b-44f6-936e-61a0cf449572 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:27.934649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a44eca-3d25-42f5-b9b8-afc7b9919b70 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:27.960938 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:27.968000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc3c41-3abb-463d-8774-c4958694a14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:27.968634 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc3c41-3abb-463d-8774-c4958694a14e None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:27.968908 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc3c41-3abb-463d-8774-c4958694a14e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:27.969176 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc3c41-3abb-463d-8774-c4958694a14e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:27.970472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc3c41-3abb-463d-8774-c4958694a14e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:27.970935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191383d7-53f1-4d0b-a5ec-af8d50631b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:27.971609 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191383d7-53f1-4d0b-a5ec-af8d50631b57 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:27.971931 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc3c41-3abb-463d-8774-c4958694a14e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:27.972035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191383d7-53f1-4d0b-a5ec-af8d50631b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:27.972284 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191383d7-53f1-4d0b-a5ec-af8d50631b57 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:27.973005 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191383d7-53f1-4d0b-a5ec-af8d50631b57 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:27.973372 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191383d7-53f1-4d0b-a5ec-af8d50631b57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:27.973831 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 126/242] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:53:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:27.975155 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc3c41-3abb-463d-8774-c4958694a14e None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:27.975697 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191383d7-53f1-4d0b-a5ec-af8d50631b57 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:27.992746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 117/243] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:27.994924 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191383d7-53f1-4d0b-a5ec-af8d50631b57 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:28.097898 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:28.103614 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191383d7-53f1-4d0b-a5ec-af8d50631b57 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:28.103914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc3c41-3abb-463d-8774-c4958694a14e None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:28.137353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4129c-2e52-470f-b7be-70edd50ef961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:28.138001 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4129c-2e52-470f-b7be-70edd50ef961 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:28.138282 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4129c-2e52-470f-b7be-70edd50ef961 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:28.138551 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4129c-2e52-470f-b7be-70edd50ef961 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:28.141053 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d6a97-a1c0-4773-8091-035b6ea1308d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:28.141662 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d6a97-a1c0-4773-8091-035b6ea1308d None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:28.141942 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d6a97-a1c0-4773-8091-035b6ea1308d None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:28.142238 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d6a97-a1c0-4773-8091-035b6ea1308d None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:28.142908 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d6a97-a1c0-4773-8091-035b6ea1308d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:28.144358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d6a97-a1c0-4773-8091-035b6ea1308d None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:28.148884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4129c-2e52-470f-b7be-70edd50ef961 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:28.154117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-31f4129c-2e52-470f-b7be-70edd50ef961 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:28.154117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:28.154117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:28.154117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:28.154117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:28.154117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:28.154117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:28.154117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:28.154117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:28.154117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:28.154117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:28.154117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:28.154117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:28.154117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:28.154117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:28.154117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:28.155051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:28.155051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:28.155051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:28.155051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:28.155051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:28.155051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:28.155051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:28.155051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:28.155051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:28.155051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:28.155051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:28.155051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:28.155051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:28.155051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:28.155051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:28.155051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:28.155879 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:28.155879 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:28.155879 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:28.155879 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:28.155879 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:28.155879 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:28.155879 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:28.155879 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:28.155879 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:28.155879 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:28.155879 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:28.155879 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:28.155879 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 127/244] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:28.156623 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4129c-2e52-470f-b7be-70edd50ef961 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:28.169605 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 118/245] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 14 01:53:27 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:53:28.169605 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf057b6c-c446-4c1b-b6df-6a09459a393e None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:28.169605 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf057b6c-c446-4c1b-b6df-6a09459a393e None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:28.169605 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf057b6c-c446-4c1b-b6df-6a09459a393e None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:28.182271 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 119/246] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:28.182271 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf057b6c-c446-4c1b-b6df-6a09459a393e None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:28.244368 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:28.299240 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4129c-2e52-470f-b7be-70edd50ef961 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:28.338723 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf057b6c-c446-4c1b-b6df-6a09459a393e None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:28.343216 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d3bc99-55f0-4e22-84cf-c86cfb60439d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:28.343997 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3bc99-55f0-4e22-84cf-c86cfb60439d None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:28.344435 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3bc99-55f0-4e22-84cf-c86cfb60439d None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:28.344872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3bc99-55f0-4e22-84cf-c86cfb60439d None admin] PATH_INFO: `/v3/users/53adce4543cf4ffe8c37426d13db3a6a` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:28.360600 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d3bc99-55f0-4e22-84cf-c86cfb60439d None admin] RBAC: Authorizing `identity:get_user(user_id=53adce4543cf4ffe8c37426d13db3a6a)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:28.362169 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d3bc99-55f0-4e22-84cf-c86cfb60439d None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:28.364071 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 128/247] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:28 2022] GET /identity/v3/users/53adce4543cf4ffe8c37426d13db3a6a => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:28.365771 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf74ee1-aebb-4ffc-910d-eba4d2678d5f None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:28.366077 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf74ee1-aebb-4ffc-910d-eba4d2678d5f None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:28.366319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf74ee1-aebb-4ffc-910d-eba4d2678d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:28.378252 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929b567-ebac-4989-b42c-e6ce2f348fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:28.378873 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929b567-ebac-4989-b42c-e6ce2f348fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:28.379150 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929b567-ebac-4989-b42c-e6ce2f348fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:28.379419 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929b567-ebac-4989-b42c-e6ce2f348fd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:28.380081 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f929b567-ebac-4989-b42c-e6ce2f348fd4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:28.380438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f929b567-ebac-4989-b42c-e6ce2f348fd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:28.383413 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f929b567-ebac-4989-b42c-e6ce2f348fd4 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:28.392004 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-dcf74ee1-aebb-4ffc-910d-eba4d2678d5f None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:28.392004 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-dcf74ee1-aebb-4ffc-910d-eba4d2678d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:28.392930 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 120/248] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:28.397677 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929b567-ebac-4989-b42c-e6ce2f348fd4 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:28.414602 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:28.472419 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 129/249] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:28.473532 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf74ee1-aebb-4ffc-910d-eba4d2678d5f None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:28.522210 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929b567-ebac-4989-b42c-e6ce2f348fd4 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:28.530912 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:28.572967 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9366f50c-b6f3-4118-ad93-b99fd73d2e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:28.574064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366f50c-b6f3-4118-ad93-b99fd73d2e82 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:28.574345 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366f50c-b6f3-4118-ad93-b99fd73d2e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:28.574612 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366f50c-b6f3-4118-ad93-b99fd73d2e82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:28.604175 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf74ee1-aebb-4ffc-910d-eba4d2678d5f None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:28.644746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68526986-76ac-46f8-b1e7-49d565cf1d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:28.645387 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68526986-76ac-46f8-b1e7-49d565cf1d60 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:28.645684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68526986-76ac-46f8-b1e7-49d565cf1d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:28.646469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68526986-76ac-46f8-b1e7-49d565cf1d60 None admin] PATH_INFO: `/v3/users/cd759a3de9ea4217a5287b1e8f6d0ba9` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:28.647642 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68526986-76ac-46f8-b1e7-49d565cf1d60 None admin] RBAC: Authorizing `identity:get_user(user_id=cd759a3de9ea4217a5287b1e8f6d0ba9)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:28.649174 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68526986-76ac-46f8-b1e7-49d565cf1d60 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:28.650829 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 130/250] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:28 2022] GET /identity/v3/users/cd759a3de9ea4217a5287b1e8f6d0ba9 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:28.652017 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3f6e7-b79e-444c-8072-cf1615989de7 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:28.652303 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3f6e7-b79e-444c-8072-cf1615989de7 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:28.652575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3f6e7-b79e-444c-8072-cf1615989de7 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:28.653779 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 131/251] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:28.657225 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3f6e7-b79e-444c-8072-cf1615989de7 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:28.673908 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9366f50c-b6f3-4118-ad93-b99fd73d2e82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:28.675955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9366f50c-b6f3-4118-ad93-b99fd73d2e82 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:28.703854 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:28.782482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3f6e7-b79e-444c-8072-cf1615989de7 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:28.800013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:28.816408 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 121/252] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:28.818932 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07177e-4c34-4021-9096-aca91f25e982 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:28.819214 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07177e-4c34-4021-9096-aca91f25e982 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:28.819581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07177e-4c34-4021-9096-aca91f25e982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:28.823879 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc725515-f041-483e-80f3-182c20da190d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:28.824514 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc725515-f041-483e-80f3-182c20da190d None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:28.824787 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc725515-f041-483e-80f3-182c20da190d None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:28.825054 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc725515-f041-483e-80f3-182c20da190d None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:28.826227 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc725515-f041-483e-80f3-182c20da190d None admin] RBAC: Authorizing `identity:get_project(project_id=60ea1e9a961b4e3b99b9fe44ac610808)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:28.827698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc725515-f041-483e-80f3-182c20da190d None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:28.829313 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 132/253] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:28 2022] GET /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:28.830955 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc725515-f041-483e-80f3-182c20da190d None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:28.839229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-ea07177e-4c34-4021-9096-aca91f25e982 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:28.840895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-ea07177e-4c34-4021-9096-aca91f25e982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:28.931158 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 122/254] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:28.932187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07177e-4c34-4021-9096-aca91f25e982 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:28.965974 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc725515-f041-483e-80f3-182c20da190d None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:29.027941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:29.037907 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12f87c-3b6d-4c2a-b469-c8c28ca70b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:29.038496 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12f87c-3b6d-4c2a-b469-c8c28ca70b27 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:29.038768 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12f87c-3b6d-4c2a-b469-c8c28ca70b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:29.039033 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12f87c-3b6d-4c2a-b469-c8c28ca70b27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:29.050973 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12f87c-3b6d-4c2a-b469-c8c28ca70b27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:29.062137 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-eb12f87c-3b6d-4c2a-b469-c8c28ca70b27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:53:29.062137 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:29.062137 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:29.062137 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:29.062137 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:29.062137 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:29.062137 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:29.062137 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:29.062137 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:29.062137 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:29.062137 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:29.062137 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:29.062137 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:29.062137 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:29.062137 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:29.062137 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:29.063095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:29.063095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:29.063095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:29.063095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:29.063095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:29.063095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:29.063095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:29.063095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:29.063095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:29.063095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:29.063095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:29.063095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:29.063095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:29.063095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:29.063095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:29.063095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:29.063924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:29.063924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:29.063924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:29.063924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:29.063924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:29.063924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:29.063924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:29.063924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:29.063924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:29.063924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:29.063924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:53:29.063924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:29.063924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 133/255] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:53:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:29.063924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12f87c-3b6d-4c2a-b469-c8c28ca70b27 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:29.099042 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07177e-4c34-4021-9096-aca91f25e982 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:29.136315 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:29.137835 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950717f3-0d14-409b-b809-5356cb392234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:29.138758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950717f3-0d14-409b-b809-5356cb392234 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:29.139317 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950717f3-0d14-409b-b809-5356cb392234 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:29.139782 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950717f3-0d14-409b-b809-5356cb392234 None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:29.141197 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950717f3-0d14-409b-b809-5356cb392234 None admin] RBAC: Authorizing `identity:get_project(project_id=60ea1e9a961b4e3b99b9fe44ac610808)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:29.142971 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950717f3-0d14-409b-b809-5356cb392234 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:29.144884 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 123/256] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:28 2022] GET /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:29.150896 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950717f3-0d14-409b-b809-5356cb392234 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:29.243722 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12f87c-3b6d-4c2a-b469-c8c28ca70b27 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:29.280034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cee35f-f1db-4f74-b15d-8863d87191c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:29.280797 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cee35f-f1db-4f74-b15d-8863d87191c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:29.280981 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cee35f-f1db-4f74-b15d-8863d87191c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:29.281245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cee35f-f1db-4f74-b15d-8863d87191c9 None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808/users/cd759a3de9ea4217a5287b1e8f6d0ba9/roles/934ee0e8483c496cb8813eccbe0ebf55` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:29.283227 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cee35f-f1db-4f74-b15d-8863d87191c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=60ea1e9a961b4e3b99b9fe44ac610808, user_id=cd759a3de9ea4217a5287b1e8f6d0ba9, role_id=934ee0e8483c496cb8813eccbe0ebf55)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:29.284704 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cee35f-f1db-4f74-b15d-8863d87191c9 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:29.291513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950717f3-0d14-409b-b809-5356cb392234 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:29.300565 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 134/257] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:53:29 2022] PUT /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808/users/cd759a3de9ea4217a5287b1e8f6d0ba9/roles/934ee0e8483c496cb8813eccbe0ebf55 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:53:29.301725 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cee35f-f1db-4f74-b15d-8863d87191c9 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:29.309215 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:29.335081 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d43789-7903-4cd0-985c-397adb2ad2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:29.335813 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d43789-7903-4cd0-985c-397adb2ad2fd None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:29.336155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d43789-7903-4cd0-985c-397adb2ad2fd None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:29.336637 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d43789-7903-4cd0-985c-397adb2ad2fd None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:29.337340 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d43789-7903-4cd0-985c-397adb2ad2fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:29.338180 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d43789-7903-4cd0-985c-397adb2ad2fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:29.339717 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d43789-7903-4cd0-985c-397adb2ad2fd None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:29.354198 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 124/258] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:29.355220 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d43789-7903-4cd0-985c-397adb2ad2fd None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:29.465590 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cee35f-f1db-4f74-b15d-8863d87191c9 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:29.467391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:29.506735 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c25d2-ceef-4728-b889-01df9894fe76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:29.508225 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c25d2-ceef-4728-b889-01df9894fe76 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:29.508700 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c25d2-ceef-4728-b889-01df9894fe76 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:29.509256 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c25d2-ceef-4728-b889-01df9894fe76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:29.510204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c25d2-ceef-4728-b889-01df9894fe76 None admin] RBAC: Adding query filter params (role.id=e5ce565bb4ca4bad8c14b5e76d2ae8ea, scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808, user.id=53adce4543cf4ffe8c37426d13db3a6a) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:29.510706 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c25d2-ceef-4728-b889-01df9894fe76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:29.512444 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c25d2-ceef-4728-b889-01df9894fe76 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:29.522947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 135/259] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:53:29 2022] GET /identity/v3/role_assignments?user.id=53adce4543cf4ffe8c37426d13db3a6a&scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808&role.id=e5ce565bb4ca4bad8c14b5e76d2ae8ea => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:29.524298 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c25d2-ceef-4728-b889-01df9894fe76 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:29.587791 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d43789-7903-4cd0-985c-397adb2ad2fd None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:29.633695 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd91a72-2c61-4195-afdd-e9e9b66900e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:29.634722 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd91a72-2c61-4195-afdd-e9e9b66900e5 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:29.635146 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd91a72-2c61-4195-afdd-e9e9b66900e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:29.635567 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd91a72-2c61-4195-afdd-e9e9b66900e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:29.649859 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd91a72-2c61-4195-afdd-e9e9b66900e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:29.655770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-7bd91a72-2c61-4195-afdd-e9e9b66900e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:29.655770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:29.655770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:29.655770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:29.655770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:29.655770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:29.655770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:29.655770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:29.655770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:29.655770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:29.655770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:29.655770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:29.655770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:29.655770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:29.655770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:29.655770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:29.657018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:29.657018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:29.657018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:29.657018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:29.657018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:29.657018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:29.657018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:29.657018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:29.657018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:29.657018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:29.657018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:29.657018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:29.657018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:29.657018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:29.657018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:29.657018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:29.658017 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:29.658017 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:29.658017 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:29.658017 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:29.658017 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:29.658017 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:29.658017 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:29.658017 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:29.658017 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:29.658017 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:29.658017 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:29.658017 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:29.659627 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 125/260] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:53:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:29.667147 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd91a72-2c61-4195-afdd-e9e9b66900e5 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:29.711270 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:29.795623 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd91a72-2c61-4195-afdd-e9e9b66900e5 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:29.808152 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c25d2-ceef-4728-b889-01df9894fe76 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:29.837328 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e49173-eb31-4006-8f8a-6845f6ad0cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:29.838059 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e49173-eb31-4006-8f8a-6845f6ad0cfe None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:29.838474 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e49173-eb31-4006-8f8a-6845f6ad0cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:29.838743 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e49173-eb31-4006-8f8a-6845f6ad0cfe None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:29.839402 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e49173-eb31-4006-8f8a-6845f6ad0cfe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:29.839699 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e49173-eb31-4006-8f8a-6845f6ad0cfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:29.841138 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e49173-eb31-4006-8f8a-6845f6ad0cfe None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:29.850490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 126/261] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:53:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:29.856685 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e49173-eb31-4006-8f8a-6845f6ad0cfe None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:29.899661 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f93fb4-9bc0-4035-84fb-1e5f6bcd890a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:29.900317 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f93fb4-9bc0-4035-84fb-1e5f6bcd890a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:29.900597 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f93fb4-9bc0-4035-84fb-1e5f6bcd890a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:29.900869 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f93fb4-9bc0-4035-84fb-1e5f6bcd890a None admin] PATH_INFO: `/v3/users/4b68f96c742f4306b09d7c0c949ad65a` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:29.902042 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f93fb4-9bc0-4035-84fb-1e5f6bcd890a None admin] RBAC: Authorizing `identity:get_user(user_id=4b68f96c742f4306b09d7c0c949ad65a)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:29.903524 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f93fb4-9bc0-4035-84fb-1e5f6bcd890a None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:29.905116 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 136/262] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:29 2022] GET /identity/v3/users/4b68f96c742f4306b09d7c0c949ad65a => generated 283 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:29.911647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f93fb4-9bc0-4035-84fb-1e5f6bcd890a None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:29.970134 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:29.990132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e49173-eb31-4006-8f8a-6845f6ad0cfe None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:30.012473 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:30.075397 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd08e38-33ac-4bee-94c2-f6f539ac3291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:30.076111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd08e38-33ac-4bee-94c2-f6f539ac3291 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:30.076339 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd08e38-33ac-4bee-94c2-f6f539ac3291 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:30.076607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd08e38-33ac-4bee-94c2-f6f539ac3291 None admin] PATH_INFO: `/v3/users/4b68f96c742f4306b09d7c0c949ad65a` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:30.082788 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd08e38-33ac-4bee-94c2-f6f539ac3291 None admin] RBAC: Authorizing `identity:get_user(user_id=4b68f96c742f4306b09d7c0c949ad65a)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:30.085166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd08e38-33ac-4bee-94c2-f6f539ac3291 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:30.086844 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 127/263] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:29 2022] GET /identity/v3/users/4b68f96c742f4306b09d7c0c949ad65a => generated 283 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:30.093102 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd08e38-33ac-4bee-94c2-f6f539ac3291 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:30.163107 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f93fb4-9bc0-4035-84fb-1e5f6bcd890a None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:30.234760 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298acb7d-2a3d-41c9-b1e8-6f2e988f42bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:30.236000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298acb7d-2a3d-41c9-b1e8-6f2e988f42bf None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:30.236477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298acb7d-2a3d-41c9-b1e8-6f2e988f42bf None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:30.236935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298acb7d-2a3d-41c9-b1e8-6f2e988f42bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:30.282891 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd08e38-33ac-4bee-94c2-f6f539ac3291 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:30.309162 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298acb7d-2a3d-41c9-b1e8-6f2e988f42bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:30.311267 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298acb7d-2a3d-41c9-b1e8-6f2e988f42bf None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:30.346404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:30.352489 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b782a9-c806-4337-957d-9cbc03ce1cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:30.353397 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b782a9-c806-4337-957d-9cbc03ce1cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:30.353681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b782a9-c806-4337-957d-9cbc03ce1cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:30.353946 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b782a9-c806-4337-957d-9cbc03ce1cc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:30.421643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 137/264] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 01:53:30.427421 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298acb7d-2a3d-41c9-b1e8-6f2e988f42bf None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:30.442410 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:30.530017 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b782a9-c806-4337-957d-9cbc03ce1cc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:30.530017 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b782a9-c806-4337-957d-9cbc03ce1cc0 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:30.574258 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298acb7d-2a3d-41c9-b1e8-6f2e988f42bf None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:30.621024 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c4e7b-b047-42be-8d16-fccbea631023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:30.622012 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c4e7b-b047-42be-8d16-fccbea631023 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:30.622564 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c4e7b-b047-42be-8d16-fccbea631023 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:30.623029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c4e7b-b047-42be-8d16-fccbea631023 None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:30.624353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c4e7b-b047-42be-8d16-fccbea631023 None admin] RBAC: Authorizing `identity:get_project(project_id=60ea1e9a961b4e3b99b9fe44ac610808)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:30.626116 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c4e7b-b047-42be-8d16-fccbea631023 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:30.628325 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 138/265] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:30 2022] GET /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:30.639337 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c4e7b-b047-42be-8d16-fccbea631023 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:30.643768 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:30.789037 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 128/266] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 697 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 01:53:30.795752 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b782a9-c806-4337-957d-9cbc03ce1cc0 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:30.832154 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c4e7b-b047-42be-8d16-fccbea631023 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:30.903264 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5998fbc-ca7b-4dd1-8d53-7c01f98d6596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:30.903731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5998fbc-ca7b-4dd1-8d53-7c01f98d6596 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:30.904012 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5998fbc-ca7b-4dd1-8d53-7c01f98d6596 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:30.904284 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5998fbc-ca7b-4dd1-8d53-7c01f98d6596 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:30.905012 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5998fbc-ca7b-4dd1-8d53-7c01f98d6596 None admin] RBAC: Adding query filter params (role.id=aa420c2fb1ab4bb3b05b004a4d89779a, scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808, user.id=4b68f96c742f4306b09d7c0c949ad65a) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:30.905318 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5998fbc-ca7b-4dd1-8d53-7c01f98d6596 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:30.906961 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5998fbc-ca7b-4dd1-8d53-7c01f98d6596 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:30.919658 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 139/267] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:53:30 2022] GET /identity/v3/role_assignments?user.id=4b68f96c742f4306b09d7c0c949ad65a&scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808&role.id=aa420c2fb1ab4bb3b05b004a4d89779a => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:30.957771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:31.012828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b782a9-c806-4337-957d-9cbc03ce1cc0 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:31.058757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5597bcb-d9be-47a2-92ec-8f9173b61655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:31.059419 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5597bcb-d9be-47a2-92ec-8f9173b61655 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:31.059697 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5597bcb-d9be-47a2-92ec-8f9173b61655 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:31.059968 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5597bcb-d9be-47a2-92ec-8f9173b61655 None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:31.066954 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5597bcb-d9be-47a2-92ec-8f9173b61655 None admin] RBAC: Authorizing `identity:get_project(project_id=60ea1e9a961b4e3b99b9fe44ac610808)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:31.068989 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5597bcb-d9be-47a2-92ec-8f9173b61655 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:31.070797 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 129/268] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:30 2022] GET /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808 => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:31.077349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5998fbc-ca7b-4dd1-8d53-7c01f98d6596 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:31.170575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:31.206436 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5998fbc-ca7b-4dd1-8d53-7c01f98d6596 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:31.240214 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf76439-a3ed-41ef-9249-9b43a134b19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:31.240822 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf76439-a3ed-41ef-9249-9b43a134b19c None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:31.241100 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf76439-a3ed-41ef-9249-9b43a134b19c None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:31.241367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf76439-a3ed-41ef-9249-9b43a134b19c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:31.242109 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf76439-a3ed-41ef-9249-9b43a134b19c None admin] RBAC: Adding query filter params (role.id=e842ac50986946a4971a51a06a39e16b, scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808, user.id=4b68f96c742f4306b09d7c0c949ad65a) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:31.242416 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf76439-a3ed-41ef-9249-9b43a134b19c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:31.243993 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf76439-a3ed-41ef-9249-9b43a134b19c None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:31.253514 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 140/269] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:53:31 2022] GET /identity/v3/role_assignments?user.id=4b68f96c742f4306b09d7c0c949ad65a&scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808&role.id=e842ac50986946a4971a51a06a39e16b => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:31.619154 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2f774-0fdc-4f97-8667-bf77298f18c0 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:31.619661 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2f774-0fdc-4f97-8667-bf77298f18c0 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:31.619767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2f774-0fdc-4f97-8667-bf77298f18c0 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:31.621032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 130/270] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:31.626361 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb047c83-ccf2-4efa-95ee-5528fb852f4b None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:31.627172 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb047c83-ccf2-4efa-95ee-5528fb852f4b None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:31.627646 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb047c83-ccf2-4efa-95ee-5528fb852f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:31.650171 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-eb047c83-ccf2-4efa-95ee-5528fb852f4b None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:31.651710 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-eb047c83-ccf2-4efa-95ee-5528fb852f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:31.718756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7608a3e-e350-45dd-b7d9-0d1f78d8cb93 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:31.719364 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7608a3e-e350-45dd-b7d9-0d1f78d8cb93 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:31.719843 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7608a3e-e350-45dd-b7d9-0d1f78d8cb93 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:31.721371 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 131/271] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:53:31.727492 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cb220-4efb-4290-8bb2-364a0b4a6034 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:31.727624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 141/272] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:53:31.728107 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cb220-4efb-4290-8bb2-364a0b4a6034 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:31.728388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cb220-4efb-4290-8bb2-364a0b4a6034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:31.747914 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-147cb220-4efb-4290-8bb2-364a0b4a6034 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:31.749624 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-147cb220-4efb-4290-8bb2-364a0b4a6034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:31.835497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 132/273] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:53:32.002183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb047c83-ccf2-4efa-95ee-5528fb852f4b None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:32.072199 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:32.102593 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587adad-4edc-4071-b1b4-208d1882101f None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:32.103225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587adad-4edc-4071-b1b4-208d1882101f None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:32.103722 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587adad-4edc-4071-b1b4-208d1882101f None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:32.105495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 133/274] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:53:32.110505 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ea7d6-553f-42c7-9df7-b9ae7aae2675 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:32.110997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ea7d6-553f-42c7-9df7-b9ae7aae2675 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:32.111462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ea7d6-553f-42c7-9df7-b9ae7aae2675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:32.115021 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb047c83-ccf2-4efa-95ee-5528fb852f4b None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:32.130843 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-566ea7d6-553f-42c7-9df7-b9ae7aae2675 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:32.132470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-566ea7d6-553f-42c7-9df7-b9ae7aae2675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:32.151909 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3218324-7c61-41e7-805b-dcfad2c74940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:32.152565 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3218324-7c61-41e7-805b-dcfad2c74940 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:32.152925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3218324-7c61-41e7-805b-dcfad2c74940 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:32.153189 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3218324-7c61-41e7-805b-dcfad2c74940 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:32.164172 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3218324-7c61-41e7-805b-dcfad2c74940 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:32.169544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-b3218324-7c61-41e7-805b-dcfad2c74940 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:32.169544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:32.169544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:32.169544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:32.169544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:32.169544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:32.169544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:32.169544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:32.169544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:32.169544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:32.169544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:32.169544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:32.169544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:32.169544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:32.169544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:32.169544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:32.170689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:32.170689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:32.170689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:32.170689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:32.170689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:32.170689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:32.170689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:32.170689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:32.170689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:32.170689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:32.170689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:32.170689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:32.170689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:32.170689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:32.170689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:32.170689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:32.171510 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:32.171510 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:32.171510 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:32.171510 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:32.171510 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:32.171510 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:32.171510 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:32.171510 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:32.171510 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:32.171510 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:32.171510 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:32.171510 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:32.171510 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 142/275] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:32.172262 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3218324-7c61-41e7-805b-dcfad2c74940 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:32.199018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:32.214052 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 134/276] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:53:32.214052 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ea7d6-553f-42c7-9df7-b9ae7aae2675 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:32.278384 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3218324-7c61-41e7-805b-dcfad2c74940 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:32.299581 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:32.315766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1665804-f1df-41a9-9a0b-db4a7c62f218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:32.316397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1665804-f1df-41a9-9a0b-db4a7c62f218 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:32.316671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1665804-f1df-41a9-9a0b-db4a7c62f218 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:32.316939 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1665804-f1df-41a9-9a0b-db4a7c62f218 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:32.318103 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1665804-f1df-41a9-9a0b-db4a7c62f218 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:32.319556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1665804-f1df-41a9-9a0b-db4a7c62f218 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:32.321163 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 143/277] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:53:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:32.327302 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1665804-f1df-41a9-9a0b-db4a7c62f218 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:32.351388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ea7d6-553f-42c7-9df7-b9ae7aae2675 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:32.402422 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7daca-88b7-45d4-90dc-5fad95ee5332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:32.403329 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7daca-88b7-45d4-90dc-5fad95ee5332 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:32.403809 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7daca-88b7-45d4-90dc-5fad95ee5332 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:32.404283 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7daca-88b7-45d4-90dc-5fad95ee5332 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:32.405169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7daca-88b7-45d4-90dc-5fad95ee5332 None admin] RBAC: Adding query filter params (name=member) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:32.405665 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7daca-88b7-45d4-90dc-5fad95ee5332 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:32.407294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7daca-88b7-45d4-90dc-5fad95ee5332 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:32.416214 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 135/278] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:32.421636 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7daca-88b7-45d4-90dc-5fad95ee5332 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:32.441407 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1665804-f1df-41a9-9a0b-db4a7c62f218 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:32.482624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5d4c1-fbaa-4f74-92c9-d3f256af1b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:32.483231 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5d4c1-fbaa-4f74-92c9-d3f256af1b8d None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:32.483508 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5d4c1-fbaa-4f74-92c9-d3f256af1b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:32.483779 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5d4c1-fbaa-4f74-92c9-d3f256af1b8d None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:32.484419 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5d4c1-fbaa-4f74-92c9-d3f256af1b8d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:32.485851 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5d4c1-fbaa-4f74-92c9-d3f256af1b8d None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:32.489435 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:32.510016 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 144/279] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 14 01:53:32 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:53:32.510016 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431dbb6a-b665-4719-864e-97eab442de53 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:32.510016 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431dbb6a-b665-4719-864e-97eab442de53 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:32.510016 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431dbb6a-b665-4719-864e-97eab442de53 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:32.511236 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 145/280] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:32.511941 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431dbb6a-b665-4719-864e-97eab442de53 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:32.557242 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7daca-88b7-45d4-90dc-5fad95ee5332 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:32.566260 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:32.598570 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef972573-cfb8-4a93-83b3-a21deb2cbf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:32.599975 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef972573-cfb8-4a93-83b3-a21deb2cbf63 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:32.600280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef972573-cfb8-4a93-83b3-a21deb2cbf63 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:32.600548 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef972573-cfb8-4a93-83b3-a21deb2cbf63 None admin] PATH_INFO: `/v3/users/3efa2868a5b641a4869a01194312b529` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:32.616992 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef972573-cfb8-4a93-83b3-a21deb2cbf63 None admin] RBAC: Authorizing `identity:get_user(user_id=3efa2868a5b641a4869a01194312b529)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:32.618536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef972573-cfb8-4a93-83b3-a21deb2cbf63 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:32.620436 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 136/281] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:32 2022] GET /identity/v3/users/3efa2868a5b641a4869a01194312b529 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:32.621643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be0eb0-1f3f-4b7f-91d2-35d72553d22c None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:32.621926 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be0eb0-1f3f-4b7f-91d2-35d72553d22c None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:32.622305 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be0eb0-1f3f-4b7f-91d2-35d72553d22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:32.637246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431dbb6a-b665-4719-864e-97eab442de53 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:32.641057 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-e1be0eb0-1f3f-4b7f-91d2-35d72553d22c None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:32.642318 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-e1be0eb0-1f3f-4b7f-91d2-35d72553d22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:32.679894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1586a16-1fe1-47c3-aefe-2d7d7c5d515c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:32.679894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1586a16-1fe1-47c3-aefe-2d7d7c5d515c None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:32.680211 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1586a16-1fe1-47c3-aefe-2d7d7c5d515c None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:32.680400 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1586a16-1fe1-47c3-aefe-2d7d7c5d515c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:32.692047 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1586a16-1fe1-47c3-aefe-2d7d7c5d515c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:32.697947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-f1586a16-1fe1-47c3-aefe-2d7d7c5d515c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:32.697947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:32.697947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:32.697947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:32.697947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:32.697947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:32.697947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:32.697947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:32.697947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:32.697947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:32.697947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:32.697947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:32.697947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:32.697947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:32.697947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:32.697947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:32.698903 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:32.698903 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:32.698903 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:32.698903 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:32.698903 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:32.698903 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:32.698903 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:32.698903 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:32.698903 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:32.698903 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:32.698903 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:32.698903 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:32.698903 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:32.698903 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:32.698903 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:32.698903 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:32.700511 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:32.700511 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:32.700511 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:32.700511 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:32.700511 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:32.700511 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:32.700511 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:32.700511 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:32.700511 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:32.700511 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:32.700511 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:32.700511 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:32.700511 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 146/282] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:32.700511 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1586a16-1fe1-47c3-aefe-2d7d7c5d515c None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:32.722761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 137/283] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:32.723125 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be0eb0-1f3f-4b7f-91d2-35d72553d22c None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:32.770543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:32.843387 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1586a16-1fe1-47c3-aefe-2d7d7c5d515c None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:32.846414 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:32.857415 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be0eb0-1f3f-4b7f-91d2-35d72553d22c None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:32.886312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06401dc6-6e20-4edc-8a68-461ead4fa81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:32.886926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06401dc6-6e20-4edc-8a68-461ead4fa81b None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:32.887203 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06401dc6-6e20-4edc-8a68-461ead4fa81b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:32.887491 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06401dc6-6e20-4edc-8a68-461ead4fa81b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:32.888199 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06401dc6-6e20-4edc-8a68-461ead4fa81b None admin] RBAC: Adding query filter params (role.id=934ee0e8483c496cb8813eccbe0ebf55, user.id=3efa2868a5b641a4869a01194312b529, scope.system=all) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:32.888504 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06401dc6-6e20-4edc-8a68-461ead4fa81b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:32.889947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06401dc6-6e20-4edc-8a68-461ead4fa81b None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:32.895196 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 147/284] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Sep 14 01:53:32 2022] GET /identity/v3/role_assignments?user.id=3efa2868a5b641a4869a01194312b529&scope.system=all&role.id=934ee0e8483c496cb8813eccbe0ebf55 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:32.904462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6e418-d9fd-421f-9834-d723c029bf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:32.905101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6e418-d9fd-421f-9834-d723c029bf3b None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:32.905381 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6e418-d9fd-421f-9834-d723c029bf3b None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:32.905650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6e418-d9fd-421f-9834-d723c029bf3b None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:32.906367 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e6e418-d9fd-421f-9834-d723c029bf3b None admin] RBAC: Adding query filter params (name=member) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:32.906671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e6e418-d9fd-421f-9834-d723c029bf3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:32.908094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e6e418-d9fd-421f-9834-d723c029bf3b None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:32.918007 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 138/285] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:32.924038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06401dc6-6e20-4edc-8a68-461ead4fa81b None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:33.015580 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6e418-d9fd-421f-9834-d723c029bf3b None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:33.045559 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06401dc6-6e20-4edc-8a68-461ead4fa81b None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:33.088832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9e7c1-c3a1-4ae8-9aea-4e41792edc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:33.089904 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9e7c1-c3a1-4ae8-9aea-4e41792edc6c None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:33.090937 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9e7c1-c3a1-4ae8-9aea-4e41792edc6c None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:33.091434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9e7c1-c3a1-4ae8-9aea-4e41792edc6c None admin] PATH_INFO: `/v3/users/cd759a3de9ea4217a5287b1e8f6d0ba9` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:33.093098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9e7c1-c3a1-4ae8-9aea-4e41792edc6c None admin] RBAC: Authorizing `identity:get_user(user_id=cd759a3de9ea4217a5287b1e8f6d0ba9)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:33.095398 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9e7c1-c3a1-4ae8-9aea-4e41792edc6c None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:33.102195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 148/286] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:32 2022] GET /identity/v3/users/cd759a3de9ea4217a5287b1e8f6d0ba9 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:33.104994 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9e7c1-c3a1-4ae8-9aea-4e41792edc6c None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:33.160208 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6e418-d9fd-421f-9834-d723c029bf3b None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:33.169163 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:33.174430 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:33.205394 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9b9b0-4825-461c-8202-bd820c999452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:33.206406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9b9b0-4825-461c-8202-bd820c999452 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:33.206898 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9b9b0-4825-461c-8202-bd820c999452 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:33.207797 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9b9b0-4825-461c-8202-bd820c999452 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:33.225703 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9b9b0-4825-461c-8202-bd820c999452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:33.234713 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9e7c1-c3a1-4ae8-9aea-4e41792edc6c None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:33.239217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-bee9b9b0-4825-461c-8202-bd820c999452 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:53:33.239217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:33.239217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:33.239217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:33.239217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:33.239217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:33.239217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:33.239217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:33.239217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:33.239217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:33.239217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:33.239217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:33.239217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:33.239217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:33.239217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:33.239217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:33.240217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:33.240217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:33.240217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:33.240217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:33.240217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:33.240217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:33.240217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:33.240217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:33.240217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:33.240217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:33.240217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:33.240217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:33.240217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:33.240217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:33.240217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:33.240217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:33.241030 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:33.241030 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:33.241030 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:33.241030 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:33.241030 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:33.241030 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:33.241030 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:33.241030 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:33.241030 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:33.241030 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:33.241030 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:53:33.241030 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:33.244853 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 139/287] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:53:33.250334 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9b9b0-4825-461c-8202-bd820c999452 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:33.296052 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2f0c2-b4e3-4222-b0c2-6890c5c96778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:33.296670 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2f0c2-b4e3-4222-b0c2-6890c5c96778 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:33.297176 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2f0c2-b4e3-4222-b0c2-6890c5c96778 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:33.297462 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2f0c2-b4e3-4222-b0c2-6890c5c96778 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:33.374532 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e2f0c2-b4e3-4222-b0c2-6890c5c96778 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:33.376228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e2f0c2-b4e3-4222-b0c2-6890c5c96778 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:33.437521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:33.461267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9b9b0-4825-461c-8202-bd820c999452 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:33.498730 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1ae5c-a947-4cab-9c88-f18af7fe7b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:33.499703 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1ae5c-a947-4cab-9c88-f18af7fe7b1e None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:33.500205 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1ae5c-a947-4cab-9c88-f18af7fe7b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:33.500676 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1ae5c-a947-4cab-9c88-f18af7fe7b1e None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:33.501547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c1ae5c-a947-4cab-9c88-f18af7fe7b1e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:33.502241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c1ae5c-a947-4cab-9c88-f18af7fe7b1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:33.503836 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c1ae5c-a947-4cab-9c88-f18af7fe7b1e None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:33.517607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 140/288] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:33.523105 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1ae5c-a947-4cab-9c88-f18af7fe7b1e None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:33.578584 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:33.588915 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 149/289] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 01:53:33.594430 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2f0c2-b4e3-4222-b0c2-6890c5c96778 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:33.670860 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1ae5c-a947-4cab-9c88-f18af7fe7b1e None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:33.715332 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0677ac-34cd-4023-b9dd-14c2071d0d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:33.716324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0677ac-34cd-4023-b9dd-14c2071d0d95 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:33.716823 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0677ac-34cd-4023-b9dd-14c2071d0d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:33.717288 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0677ac-34cd-4023-b9dd-14c2071d0d95 None admin] PATH_INFO: `/v3/users/53adce4543cf4ffe8c37426d13db3a6a` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:33.718697 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0677ac-34cd-4023-b9dd-14c2071d0d95 None admin] RBAC: Authorizing `identity:get_user(user_id=53adce4543cf4ffe8c37426d13db3a6a)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:33.720381 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0677ac-34cd-4023-b9dd-14c2071d0d95 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:33.722605 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 141/290] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:33 2022] GET /identity/v3/users/53adce4543cf4ffe8c37426d13db3a6a => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:33.728457 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0677ac-34cd-4023-b9dd-14c2071d0d95 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:33.757574 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2f0c2-b4e3-4222-b0c2-6890c5c96778 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:33.796688 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae5d2d-a3f6-4d46-a9cf-db459a448158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:33.797665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae5d2d-a3f6-4d46-a9cf-db459a448158 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:33.798619 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae5d2d-a3f6-4d46-a9cf-db459a448158 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:33.799142 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae5d2d-a3f6-4d46-a9cf-db459a448158 None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:33.800462 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae5d2d-a3f6-4d46-a9cf-db459a448158 None admin] RBAC: Authorizing `identity:get_project(project_id=60ea1e9a961b4e3b99b9fe44ac610808)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:33.802711 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae5d2d-a3f6-4d46-a9cf-db459a448158 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:33.804847 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 150/291] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:33 2022] GET /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:33.810405 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae5d2d-a3f6-4d46-a9cf-db459a448158 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:33.815777 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:33.955633 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:33.964880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0677ac-34cd-4023-b9dd-14c2071d0d95 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:33.997052 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae5d2d-a3f6-4d46-a9cf-db459a448158 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:34.011252 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020ef55-f1af-4f7c-8e40-cf1afc74f9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:34.011885 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020ef55-f1af-4f7c-8e40-cf1afc74f9cb None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:34.012159 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020ef55-f1af-4f7c-8e40-cf1afc74f9cb None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:34.012424 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020ef55-f1af-4f7c-8e40-cf1afc74f9cb None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:34.013536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020ef55-f1af-4f7c-8e40-cf1afc74f9cb None admin] RBAC: Authorizing `identity:get_project(project_id=60ea1e9a961b4e3b99b9fe44ac610808)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:34.015005 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020ef55-f1af-4f7c-8e40-cf1afc74f9cb None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:34.017033 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 142/292] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:33 2022] GET /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808 => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:34.022787 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020ef55-f1af-4f7c-8e40-cf1afc74f9cb None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:34.048530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72015ce-766e-4820-9672-bbe6dd9fe2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:34.049246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72015ce-766e-4820-9672-bbe6dd9fe2bf None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:34.049524 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72015ce-766e-4820-9672-bbe6dd9fe2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:34.049799 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72015ce-766e-4820-9672-bbe6dd9fe2bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:34.050548 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72015ce-766e-4820-9672-bbe6dd9fe2bf None admin] RBAC: Adding query filter params (role.id=934ee0e8483c496cb8813eccbe0ebf55, scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808, user.id=cd759a3de9ea4217a5287b1e8f6d0ba9) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:34.050854 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72015ce-766e-4820-9672-bbe6dd9fe2bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:34.052374 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72015ce-766e-4820-9672-bbe6dd9fe2bf None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:34.058647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 151/293] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:53:33 2022] GET /identity/v3/role_assignments?user.id=cd759a3de9ea4217a5287b1e8f6d0ba9&scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808&role.id=934ee0e8483c496cb8813eccbe0ebf55 => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:34.158027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020ef55-f1af-4f7c-8e40-cf1afc74f9cb None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:34.195680 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d78250-16a7-4430-8696-30c2f4641a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:34.196365 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d78250-16a7-4430-8696-30c2f4641a15 None admin] REQUEST_METHOD: `PUT` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:34.196644 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d78250-16a7-4430-8696-30c2f4641a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:34.196917 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d78250-16a7-4430-8696-30c2f4641a15 None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808/users/53adce4543cf4ffe8c37426d13db3a6a/roles/e5ce565bb4ca4bad8c14b5e76d2ae8ea` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:34.198949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d78250-16a7-4430-8696-30c2f4641a15 None admin] RBAC: Authorizing `identity:create_grant(project_id=60ea1e9a961b4e3b99b9fe44ac610808, user_id=53adce4543cf4ffe8c37426d13db3a6a, role_id=e5ce565bb4ca4bad8c14b5e76d2ae8ea)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:34.200424 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d78250-16a7-4430-8696-30c2f4641a15 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:34.216326 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 143/294] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:53:34 2022] PUT /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808/users/53adce4543cf4ffe8c37426d13db3a6a/roles/e5ce565bb4ca4bad8c14b5e76d2ae8ea => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:53:35.150017 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73f01b-8d1b-4d4d-ba58-249409d75afe None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:35.151072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73f01b-8d1b-4d4d-ba58-249409d75afe None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:35.151582 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73f01b-8d1b-4d4d-ba58-249409d75afe None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:35.153128 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 152/295] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:35.158695 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4382b4-4b33-4783-ab9b-279229cd607a None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:35.159556 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4382b4-4b33-4783-ab9b-279229cd607a None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:35.160028 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4382b4-4b33-4783-ab9b-279229cd607a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:35.180304 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-9e4382b4-4b33-4783-ab9b-279229cd607a None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:35.181863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-9e4382b4-4b33-4783-ab9b-279229cd607a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:35.202029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:35.256022 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 144/296] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:35.322763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41aa8c-313b-4a1f-91fd-f5be63e4685b None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:35.323123 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41aa8c-313b-4a1f-91fd-f5be63e4685b None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:35.323308 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41aa8c-313b-4a1f-91fd-f5be63e4685b None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:35.324993 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 153/297] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:35.330662 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1139642-2112-48dd-8511-dfef9ea64df3 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:35.331268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1139642-2112-48dd-8511-dfef9ea64df3 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:35.331764 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1139642-2112-48dd-8511-dfef9ea64df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:35.351367 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-b1139642-2112-48dd-8511-dfef9ea64df3 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:35.352884 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-b1139642-2112-48dd-8511-dfef9ea64df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:35.430140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 145/298] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:35.433793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c6bf4-9a83-4e50-909d-752ecd0344b1 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:35.434345 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c6bf4-9a83-4e50-909d-752ecd0344b1 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:35.434817 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c6bf4-9a83-4e50-909d-752ecd0344b1 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:35.436363 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 154/299] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:35.442068 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9d8aa-4174-4aea-b8b1-52db612ee4e6 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:35.442068 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9d8aa-4174-4aea-b8b1-52db612ee4e6 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:35.442263 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9d8aa-4174-4aea-b8b1-52db612ee4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:35.459918 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-def9d8aa-4174-4aea-b8b1-52db612ee4e6 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:35.461098 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-def9d8aa-4174-4aea-b8b1-52db612ee4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:35.526723 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 146/300] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:35.539049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c6bf4-9a83-4e50-909d-752ecd0344b1 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:35.608310 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:35.644523 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c6bf4-9a83-4e50-909d-752ecd0344b1 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:35.678523 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb75047-ecb1-4830-8ab9-abf76f11556c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:35.679144 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb75047-ecb1-4830-8ab9-abf76f11556c None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:35.679425 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb75047-ecb1-4830-8ab9-abf76f11556c None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:35.679694 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb75047-ecb1-4830-8ab9-abf76f11556c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:35.689698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb75047-ecb1-4830-8ab9-abf76f11556c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:35.695575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-abb75047-ecb1-4830-8ab9-abf76f11556c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:53:35.695575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:35.695575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:35.695575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:35.695575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:35.695575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:35.695575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:35.695575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:35.695575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:35.695575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:35.695575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:35.695575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:35.695575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:35.695575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:35.695575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:35.695575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:35.696665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:35.696665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:35.696665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:35.696665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:35.696665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:35.696665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:35.696665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:35.696665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:35.696665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:35.696665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:35.696665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:35.696665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:35.696665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:35.696665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:35.696665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:35.696665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:35.697645 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:35.697645 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:35.697645 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:35.697645 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:35.697645 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:35.697645 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:35.697645 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:35.697645 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:35.697645 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:35.697645 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:35.697645 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:53:35.697645 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:35.699471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 155/301] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:35.700328 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9d8aa-4174-4aea-b8b1-52db612ee4e6 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:35.704870 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb75047-ecb1-4830-8ab9-abf76f11556c None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:35.742332 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:35.815969 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9d8aa-4174-4aea-b8b1-52db612ee4e6 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:35.818111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb75047-ecb1-4830-8ab9-abf76f11556c None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:35.845771 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:35.853389 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0833ad71-185e-4ea9-81fd-ca8d363ff642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:35.854043 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0833ad71-185e-4ea9-81fd-ca8d363ff642 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:35.854304 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b4011-3404-49a5-a429-02b4748a2ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:35.854419 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0833ad71-185e-4ea9-81fd-ca8d363ff642 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:35.854611 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0833ad71-185e-4ea9-81fd-ca8d363ff642 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:35.854910 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b4011-3404-49a5-a429-02b4748a2ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:35.855192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b4011-3404-49a5-a429-02b4748a2ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:35.855466 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b4011-3404-49a5-a429-02b4748a2ce2 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:35.856122 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b4011-3404-49a5-a429-02b4748a2ce2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:35.856430 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b4011-3404-49a5-a429-02b4748a2ce2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:35.857808 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b4011-3404-49a5-a429-02b4748a2ce2 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:35.865505 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0833ad71-185e-4ea9-81fd-ca8d363ff642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:35.867056 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 156/302] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:35.867840 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b4011-3404-49a5-a429-02b4748a2ce2 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:35.870958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-0833ad71-185e-4ea9-81fd-ca8d363ff642 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:35.870958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:35.870958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:35.870958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:35.870958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:35.870958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:35.870958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:35.870958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:35.870958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:35.870958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:35.870958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:35.870958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:35.870958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:35.870958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:35.870958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:35.870958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:35.872056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:35.872056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:35.872056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:35.872056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:35.872056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:35.872056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:35.872056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:35.872056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:35.872056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:35.872056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:35.872056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:35.872056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:35.872056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:35.872056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:35.872056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:35.872056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:35.873134 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:35.873134 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:35.873134 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:35.873134 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:35.873134 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:35.873134 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:35.873134 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:35.873134 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:35.873134 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:35.873134 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:35.873134 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:35.873134 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:35.876689 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 147/303] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:35.877501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0833ad71-185e-4ea9-81fd-ca8d363ff642 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:35.957470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:35.974305 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b4011-3404-49a5-a429-02b4748a2ce2 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:35.985401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0833ad71-185e-4ea9-81fd-ca8d363ff642 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:36.008237 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17837a3-6772-4594-8863-1a3c0cfeb3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:36.008867 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17837a3-6772-4594-8863-1a3c0cfeb3ea None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:36.009149 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17837a3-6772-4594-8863-1a3c0cfeb3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:36.009419 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17837a3-6772-4594-8863-1a3c0cfeb3ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:36.010548 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17837a3-6772-4594-8863-1a3c0cfeb3ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:36.011985 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17837a3-6772-4594-8863-1a3c0cfeb3ea None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:36.013838 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 157/304] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:53:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:36.014502 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17837a3-6772-4594-8863-1a3c0cfeb3ea None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:36.031253 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f1140-2dff-4c37-8626-29f48f9b3594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:36.031896 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f1140-2dff-4c37-8626-29f48f9b3594 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:36.032197 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f1140-2dff-4c37-8626-29f48f9b3594 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:36.032449 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f1140-2dff-4c37-8626-29f48f9b3594 None admin] PATH_INFO: `/v3/users/9442e5a46ace4cc3a925f434519c0ae2` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:36.046735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f1140-2dff-4c37-8626-29f48f9b3594 None admin] RBAC: Authorizing `identity:get_user(user_id=9442e5a46ace4cc3a925f434519c0ae2)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:36.048671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f1140-2dff-4c37-8626-29f48f9b3594 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:36.055618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 148/305] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:35 2022] GET /identity/v3/users/9442e5a46ace4cc3a925f434519c0ae2 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:36.055618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f1140-2dff-4c37-8626-29f48f9b3594 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:36.096354 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:36.147271 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17837a3-6772-4594-8863-1a3c0cfeb3ea None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:36.161478 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f1140-2dff-4c37-8626-29f48f9b3594 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:36.185915 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0c265-124f-4c59-8bb3-33a9ac8b8036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:36.186761 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0c265-124f-4c59-8bb3-33a9ac8b8036 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:36.187067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0c265-124f-4c59-8bb3-33a9ac8b8036 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:36.187445 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0c265-124f-4c59-8bb3-33a9ac8b8036 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:36.188174 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae0c265-124f-4c59-8bb3-33a9ac8b8036 None admin] RBAC: Adding query filter params (name=member) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:36.188547 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae0c265-124f-4c59-8bb3-33a9ac8b8036 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:36.190012 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae0c265-124f-4c59-8bb3-33a9ac8b8036 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:36.193058 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:36.203164 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 158/306] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:36.203325 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bd5b7-82bf-478f-bc75-deb13a18c53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:36.203433 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0c265-124f-4c59-8bb3-33a9ac8b8036 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:36.203561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bd5b7-82bf-478f-bc75-deb13a18c53f None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:36.203561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bd5b7-82bf-478f-bc75-deb13a18c53f None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:36.203561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bd5b7-82bf-478f-bc75-deb13a18c53f None admin] PATH_INFO: `/v3/groups` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:36.203561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bd5b7-82bf-478f-bc75-deb13a18c53f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:36.203561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bd5b7-82bf-478f-bc75-deb13a18c53f None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:36.218867 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 149/307] 10.222.0.15 () {60 vars in 1134 bytes} [Wed Sep 14 01:53:36 2022] POST /identity/v3/groups => generated 229 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:53:36.231287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bd5b7-82bf-478f-bc75-deb13a18c53f None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:36.326573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0c265-124f-4c59-8bb3-33a9ac8b8036 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:36.352138 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:36.371113 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1221a75-83d3-4e26-b572-b3a6fc0d5aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:36.371913 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1221a75-83d3-4e26-b572-b3a6fc0d5aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:36.372289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1221a75-83d3-4e26-b572-b3a6fc0d5aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:36.372670 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1221a75-83d3-4e26-b572-b3a6fc0d5aa4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:36.373499 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1221a75-83d3-4e26-b572-b3a6fc0d5aa4 None admin] RBAC: Adding query filter params (role.id=e5ce565bb4ca4bad8c14b5e76d2ae8ea, user.id=9442e5a46ace4cc3a925f434519c0ae2, scope.system=all) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:36.374497 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1221a75-83d3-4e26-b572-b3a6fc0d5aa4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:36.375930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1221a75-83d3-4e26-b572-b3a6fc0d5aa4 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:36.390592 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bd5b7-82bf-478f-bc75-deb13a18c53f None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:36.400990 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 159/308] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Sep 14 01:53:36 2022] GET /identity/v3/role_assignments?user.id=9442e5a46ace4cc3a925f434519c0ae2&scope.system=all&role.id=e5ce565bb4ca4bad8c14b5e76d2ae8ea => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:36.435320 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147b545-fcb1-4043-b6c1-639e9c55225d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:36.436396 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147b545-fcb1-4043-b6c1-639e9c55225d None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:36.436883 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147b545-fcb1-4043-b6c1-639e9c55225d None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:36.437360 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147b545-fcb1-4043-b6c1-639e9c55225d None admin] PATH_INFO: `/v3/users/3efa2868a5b641a4869a01194312b529` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:36.438958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8147b545-fcb1-4043-b6c1-639e9c55225d None admin] RBAC: Authorizing `identity:get_user(user_id=3efa2868a5b641a4869a01194312b529)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:36.440762 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8147b545-fcb1-4043-b6c1-639e9c55225d None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:36.443250 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 150/309] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:36 2022] GET /identity/v3/users/3efa2868a5b641a4869a01194312b529 => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:36.449771 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1221a75-83d3-4e26-b572-b3a6fc0d5aa4 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:36.542041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4266bc-58ea-45cf-b031-dd65ccc93299 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:36.542341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4266bc-58ea-45cf-b031-dd65ccc93299 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:36.542505 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4266bc-58ea-45cf-b031-dd65ccc93299 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:36.544004 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 151/310] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:53:36.549761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7cec8-8cdf-415c-938b-41dd904d7372 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:36.550400 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7cec8-8cdf-415c-938b-41dd904d7372 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:36.550889 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7cec8-8cdf-415c-938b-41dd904d7372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:36.571462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-61b7cec8-8cdf-415c-938b-41dd904d7372 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:36.573150 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-61b7cec8-8cdf-415c-938b-41dd904d7372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:36.574658 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1221a75-83d3-4e26-b572-b3a6fc0d5aa4 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:36.615763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2d831-e487-4caf-a488-6cdc3774de6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:36.616532 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2d831-e487-4caf-a488-6cdc3774de6f None admin] REQUEST_METHOD: `PUT` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:36.616818 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2d831-e487-4caf-a488-6cdc3774de6f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:36.617185 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2d831-e487-4caf-a488-6cdc3774de6f None admin] PATH_INFO: `/v3/system/users/3efa2868a5b641a4869a01194312b529/roles/934ee0e8483c496cb8813eccbe0ebf55` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:36.618655 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2d831-e487-4caf-a488-6cdc3774de6f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3efa2868a5b641a4869a01194312b529, role_id=934ee0e8483c496cb8813eccbe0ebf55)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:36.620119 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2d831-e487-4caf-a488-6cdc3774de6f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:36.635633 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 160/311] 10.222.0.15 () {58 vars in 1334 bytes} [Wed Sep 14 01:53:36 2022] PUT /identity/v3/system/users/3efa2868a5b641a4869a01194312b529/roles/934ee0e8483c496cb8813eccbe0ebf55 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:53:36.648760 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:36.664474 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 152/312] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:53:36.951687 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2d831-e487-4caf-a488-6cdc3774de6f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:37.003683 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:37.068132 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2d831-e487-4caf-a488-6cdc3774de6f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:37.103621 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7c021-8bed-4d78-af48-3965abdcba08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:37.104546 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7c021-8bed-4d78-af48-3965abdcba08 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:37.104827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7c021-8bed-4d78-af48-3965abdcba08 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:37.105095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7c021-8bed-4d78-af48-3965abdcba08 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:37.118124 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7c021-8bed-4d78-af48-3965abdcba08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:37.121954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-5cf7c021-8bed-4d78-af48-3965abdcba08 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:53:37.121954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:37.121954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:37.121954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:37.121954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:37.121954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:37.121954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:37.121954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:37.121954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:37.121954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:37.121954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:37.121954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:37.121954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:37.121954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:37.121954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:37.121954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:37.124067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:37.124067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:37.124067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:37.124067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:37.124067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:37.124067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:37.124067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:37.124067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:37.124067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:37.124067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:37.124067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:37.124067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:37.124067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:37.124067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:37.124067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:37.124067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:37.124912 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:37.124912 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:37.124912 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:37.124912 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:37.124912 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:37.124912 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:37.124912 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:37.124912 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:37.124912 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:37.124912 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:37.124912 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:53:37.124912 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:37.124912 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 161/313] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:37.128266 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7cec8-8cdf-415c-938b-41dd904d7372 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:37.236027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7cec8-8cdf-415c-938b-41dd904d7372 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:37.271618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10946bcf-c4af-4cec-af02-55b4bb09c2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:37.272231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10946bcf-c4af-4cec-af02-55b4bb09c2ad None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:37.272507 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10946bcf-c4af-4cec-af02-55b4bb09c2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:37.272774 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10946bcf-c4af-4cec-af02-55b4bb09c2ad None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:37.273426 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10946bcf-c4af-4cec-af02-55b4bb09c2ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:37.273721 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10946bcf-c4af-4cec-af02-55b4bb09c2ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:37.275121 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10946bcf-c4af-4cec-af02-55b4bb09c2ad None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:37.283742 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 153/314] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:37.289905 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7c021-8bed-4d78-af48-3965abdcba08 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:37.400846 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7c021-8bed-4d78-af48-3965abdcba08 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:37.416343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:37.446116 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf54c92-1494-474b-b4e5-f378969b628f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:37.446912 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf54c92-1494-474b-b4e5-f378969b628f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:37.447185 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf54c92-1494-474b-b4e5-f378969b628f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:37.447451 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf54c92-1494-474b-b4e5-f378969b628f None admin] PATH_INFO: `/v3/users/53adce4543cf4ffe8c37426d13db3a6a` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:37.448892 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf54c92-1494-474b-b4e5-f378969b628f None admin] RBAC: Authorizing `identity:get_user(user_id=53adce4543cf4ffe8c37426d13db3a6a)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:37.450366 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf54c92-1494-474b-b4e5-f378969b628f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:37.452238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 162/315] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:37 2022] GET /identity/v3/users/53adce4543cf4ffe8c37426d13db3a6a => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:37.457392 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10946bcf-c4af-4cec-af02-55b4bb09c2ad None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:37.514139 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:37.562577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10946bcf-c4af-4cec-af02-55b4bb09c2ad None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:37.599726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7945435-3d19-4de4-be8b-3040695c5e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:37.600333 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7945435-3d19-4de4-be8b-3040695c5e43 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:37.600608 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7945435-3d19-4de4-be8b-3040695c5e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:37.600873 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7945435-3d19-4de4-be8b-3040695c5e43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:37.669464 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7945435-3d19-4de4-be8b-3040695c5e43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:37.671101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7945435-3d19-4de4-be8b-3040695c5e43 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:37.730397 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:37.775684 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 154/316] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:37.780962 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf54c92-1494-474b-b4e5-f378969b628f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:37.883083 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf54c92-1494-474b-b4e5-f378969b628f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:37.918769 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4f0f6-c7f6-4630-bb0c-bd85230cc830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:37.919398 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4f0f6-c7f6-4630-bb0c-bd85230cc830 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:37.919676 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4f0f6-c7f6-4630-bb0c-bd85230cc830 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:37.919946 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4f0f6-c7f6-4630-bb0c-bd85230cc830 None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:37.921063 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c4f0f6-c7f6-4630-bb0c-bd85230cc830 None admin] RBAC: Authorizing `identity:get_project(project_id=60ea1e9a961b4e3b99b9fe44ac610808)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:37.922522 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c4f0f6-c7f6-4630-bb0c-bd85230cc830 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:37.924420 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 163/317] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:37 2022] GET /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:37.930094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7945435-3d19-4de4-be8b-3040695c5e43 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:38.032792 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7945435-3d19-4de4-be8b-3040695c5e43 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:38.069637 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4d1ee-394a-47b6-9ef0-6fdd46fdb051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:38.070843 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4d1ee-394a-47b6-9ef0-6fdd46fdb051 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:38.071334 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4d1ee-394a-47b6-9ef0-6fdd46fdb051 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:38.071815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4d1ee-394a-47b6-9ef0-6fdd46fdb051 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:38.072572 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4d1ee-394a-47b6-9ef0-6fdd46fdb051 None admin] RBAC: Adding query filter params (role.id=e5ce565bb4ca4bad8c14b5e76d2ae8ea, scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808, user.id=53adce4543cf4ffe8c37426d13db3a6a) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:38.072877 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4d1ee-394a-47b6-9ef0-6fdd46fdb051 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:38.074357 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4d1ee-394a-47b6-9ef0-6fdd46fdb051 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:38.081088 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 155/318] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:53:37 2022] GET /identity/v3/role_assignments?user.id=53adce4543cf4ffe8c37426d13db3a6a&scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808&role.id=e5ce565bb4ca4bad8c14b5e76d2ae8ea => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:38.575878 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9d6af-405a-4c2e-9add-5be642f66c37 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:38.576388 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9d6af-405a-4c2e-9add-5be642f66c37 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:38.576388 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9d6af-405a-4c2e-9add-5be642f66c37 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:38.577788 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 164/319] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:38.583388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a3df7-4e3a-48f7-b19a-aafd8b36ec79 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:38.583766 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a3df7-4e3a-48f7-b19a-aafd8b36ec79 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:38.583938 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a3df7-4e3a-48f7-b19a-aafd8b36ec79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:38.601897 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-bc1a3df7-4e3a-48f7-b19a-aafd8b36ec79 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:38.603092 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-bc1a3df7-4e3a-48f7-b19a-aafd8b36ec79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:38.614130 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:38.631361 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280696b-dc48-41c2-b036-077d307e7bc4 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:38.631653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280696b-dc48-41c2-b036-077d307e7bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:38.631918 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280696b-dc48-41c2-b036-077d307e7bc4 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:38.633506 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 165/320] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:53:38.638230 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8f3f7-5979-4c26-87ee-54ece74d9e90 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:38.638457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8f3f7-5979-4c26-87ee-54ece74d9e90 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:38.638734 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8f3f7-5979-4c26-87ee-54ece74d9e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:38.659184 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-26e8f3f7-5979-4c26-87ee-54ece74d9e90 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:38.660370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-26e8f3f7-5979-4c26-87ee-54ece74d9e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:38.672559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 156/321] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:38.703988 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:38.731904 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 166/322] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:53:38.872731 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b71474-7093-49bd-9b8d-7fa04e046903 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:38.873037 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b71474-7093-49bd-9b8d-7fa04e046903 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:38.873291 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b71474-7093-49bd-9b8d-7fa04e046903 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:38.874801 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 157/323] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:38.880504 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c2098-d86c-429a-b1b0-1d01499be816 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:38.880867 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c2098-d86c-429a-b1b0-1d01499be816 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:38.881204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c2098-d86c-429a-b1b0-1d01499be816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:38.899544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-b51c2098-d86c-429a-b1b0-1d01499be816 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:38.900914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-b51c2098-d86c-429a-b1b0-1d01499be816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:38.938166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b71474-7093-49bd-9b8d-7fa04e046903 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:38.974040 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 167/324] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:39.005277 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c2098-d86c-429a-b1b0-1d01499be816 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:39.052237 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b71474-7093-49bd-9b8d-7fa04e046903 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:39.087304 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff8ee0-2fb7-400c-8e62-ffa3d68482d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:39.087934 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff8ee0-2fb7-400c-8e62-ffa3d68482d1 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:39.088207 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff8ee0-2fb7-400c-8e62-ffa3d68482d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:39.088471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff8ee0-2fb7-400c-8e62-ffa3d68482d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:39.089549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff8ee0-2fb7-400c-8e62-ffa3d68482d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:39.091012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff8ee0-2fb7-400c-8e62-ffa3d68482d1 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:39.092556 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 158/325] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:53:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:39.097946 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff8ee0-2fb7-400c-8e62-ffa3d68482d1 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:39.108494 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c2098-d86c-429a-b1b0-1d01499be816 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:39.112224 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:39.114306 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:39.145960 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abdf7f-dbac-4a15-8318-2d61d90d2398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:39.146718 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abdf7f-dbac-4a15-8318-2d61d90d2398 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:39.146895 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abdf7f-dbac-4a15-8318-2d61d90d2398 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:39.147163 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abdf7f-dbac-4a15-8318-2d61d90d2398 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:39.157832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99abdf7f-dbac-4a15-8318-2d61d90d2398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:39.162650 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-99abdf7f-dbac-4a15-8318-2d61d90d2398 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:53:39.162650 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:39.162650 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:39.162650 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:39.162650 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:39.162650 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:39.162650 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:39.162650 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:39.162650 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:39.162650 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:39.162650 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:39.162650 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:39.162650 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:39.162650 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:39.162650 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:39.162650 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:39.163559 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:39.163559 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:39.163559 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:39.163559 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:39.163559 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:39.163559 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:39.163559 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:39.163559 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:39.163559 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:39.163559 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:39.163559 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:39.163559 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:39.163559 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:39.163559 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:39.163559 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:39.163559 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:39.164370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:39.164370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:39.164370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:39.164370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:39.164370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:39.164370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:39.164370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:39.164370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:39.164370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:39.164370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:39.164370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:53:39.164370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:39.164370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 168/326] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:53:39.169265 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abdf7f-dbac-4a15-8318-2d61d90d2398 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:39.209874 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff8ee0-2fb7-400c-8e62-ffa3d68482d1 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:39.245020 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fe85b-3cee-46d3-8a3f-fec9bf1ec562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:39.245620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fe85b-3cee-46d3-8a3f-fec9bf1ec562 None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:39.245896 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fe85b-3cee-46d3-8a3f-fec9bf1ec562 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:39.246193 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fe85b-3cee-46d3-8a3f-fec9bf1ec562 None admin] PATH_INFO: `/v3/groups` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:39.246812 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363fe85b-3cee-46d3-8a3f-fec9bf1ec562 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:39.248245 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363fe85b-3cee-46d3-8a3f-fec9bf1ec562 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:39.263056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 159/327] 10.222.0.15 () {60 vars in 1134 bytes} [Wed Sep 14 01:53:39 2022] POST /identity/v3/groups => generated 226 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:53:39.263862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fe85b-3cee-46d3-8a3f-fec9bf1ec562 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:39.276352 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abdf7f-dbac-4a15-8318-2d61d90d2398 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:39.313923 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e155b-fab7-43bb-aec3-2bdc9039202b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:39.314580 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e155b-fab7-43bb-aec3-2bdc9039202b None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:39.314862 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e155b-fab7-43bb-aec3-2bdc9039202b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:39.315131 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e155b-fab7-43bb-aec3-2bdc9039202b None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:39.315806 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e155b-fab7-43bb-aec3-2bdc9039202b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:39.316111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e155b-fab7-43bb-aec3-2bdc9039202b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:39.318172 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e155b-fab7-43bb-aec3-2bdc9039202b None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:39.327753 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 169/328] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:39.333319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e155b-fab7-43bb-aec3-2bdc9039202b None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:39.349296 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:39.366727 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:39.381043 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fe85b-3cee-46d3-8a3f-fec9bf1ec562 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:39.427352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb5d4b5-7630-4413-a094-1a63d28e5785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:39.428289 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5d4b5-7630-4413-a094-1a63d28e5785 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:39.428618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5d4b5-7630-4413-a094-1a63d28e5785 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:39.429155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5d4b5-7630-4413-a094-1a63d28e5785 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:39.446056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb5d4b5-7630-4413-a094-1a63d28e5785 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:39.451490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-bcb5d4b5-7630-4413-a094-1a63d28e5785 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:39.451490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:39.451490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:39.451490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:39.451490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:39.451490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:39.451490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:39.451490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:39.451490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:39.451490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:39.451490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:39.451490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:39.451490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:39.451490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:39.451490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:39.451490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:39.452406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:39.452406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:39.452406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:39.452406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:39.452406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:39.452406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:39.452406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:39.452406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:39.452406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:39.452406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:39.452406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:39.452406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:39.452406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:39.452406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:39.452406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:39.452406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:39.453218 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:39.453218 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:39.453218 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:39.453218 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:39.453218 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:39.453218 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:39.453218 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:39.453218 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:39.453218 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:39.453218 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:39.453218 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:39.453218 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:39.453218 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 160/329] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:39.458966 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb5d4b5-7630-4413-a094-1a63d28e5785 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:39.481675 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e155b-fab7-43bb-aec3-2bdc9039202b None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:39.518665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cdffd-c61e-4e7e-b05d-308bfc80b5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:39.519300 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cdffd-c61e-4e7e-b05d-308bfc80b5be None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:39.519587 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cdffd-c61e-4e7e-b05d-308bfc80b5be None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:39.520553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cdffd-c61e-4e7e-b05d-308bfc80b5be None admin] PATH_INFO: `/v3/users/9442e5a46ace4cc3a925f434519c0ae2` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:39.521513 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cdffd-c61e-4e7e-b05d-308bfc80b5be None admin] RBAC: Authorizing `identity:get_user(user_id=9442e5a46ace4cc3a925f434519c0ae2)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:39.522966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cdffd-c61e-4e7e-b05d-308bfc80b5be None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:39.526047 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 170/330] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:39 2022] GET /identity/v3/users/9442e5a46ace4cc3a925f434519c0ae2 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:39.530270 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cdffd-c61e-4e7e-b05d-308bfc80b5be None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:39.578701 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb5d4b5-7630-4413-a094-1a63d28e5785 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:39.625931 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18df8a-8795-46a9-9796-b82529b0998c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:39.626584 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18df8a-8795-46a9-9796-b82529b0998c None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:39.626829 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18df8a-8795-46a9-9796-b82529b0998c None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:39.627096 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18df8a-8795-46a9-9796-b82529b0998c None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:39.628227 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18df8a-8795-46a9-9796-b82529b0998c None admin] RBAC: Adding query filter params (name=member) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:39.628705 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18df8a-8795-46a9-9796-b82529b0998c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:39.630649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18df8a-8795-46a9-9796-b82529b0998c None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:39.632980 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:39.642850 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 161/331] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:39.649186 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18df8a-8795-46a9-9796-b82529b0998c None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:39.655308 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:39.664545 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cdffd-c61e-4e7e-b05d-308bfc80b5be None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:39.709127 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf3167-d33f-4e5d-a473-ee02a721aea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:39.710205 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf3167-d33f-4e5d-a473-ee02a721aea1 None admin] REQUEST_METHOD: `PUT` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:39.710719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf3167-d33f-4e5d-a473-ee02a721aea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:39.711233 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf3167-d33f-4e5d-a473-ee02a721aea1 None admin] PATH_INFO: `/v3/system/users/9442e5a46ace4cc3a925f434519c0ae2/roles/e5ce565bb4ca4bad8c14b5e76d2ae8ea` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:39.713562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf3167-d33f-4e5d-a473-ee02a721aea1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9442e5a46ace4cc3a925f434519c0ae2, role_id=e5ce565bb4ca4bad8c14b5e76d2ae8ea)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:39.715457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf3167-d33f-4e5d-a473-ee02a721aea1 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:39.727201 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 171/332] 10.222.0.15 () {58 vars in 1334 bytes} [Wed Sep 14 01:53:39 2022] PUT /identity/v3/system/users/9442e5a46ace4cc3a925f434519c0ae2/roles/e5ce565bb4ca4bad8c14b5e76d2ae8ea => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:53:39.784363 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18df8a-8795-46a9-9796-b82529b0998c None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:39.832591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4b9e4-9225-4779-a26c-4c5a4cf3bc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:39.832591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4b9e4-9225-4779-a26c-4c5a4cf3bc0c None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:39.832915 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4b9e4-9225-4779-a26c-4c5a4cf3bc0c None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:39.833207 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4b9e4-9225-4779-a26c-4c5a4cf3bc0c None admin] PATH_INFO: `/v3/users/3efa2868a5b641a4869a01194312b529` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:39.834436 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4b9e4-9225-4779-a26c-4c5a4cf3bc0c None admin] RBAC: Authorizing `identity:get_user(user_id=3efa2868a5b641a4869a01194312b529)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:39.835958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4b9e4-9225-4779-a26c-4c5a4cf3bc0c None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:39.838122 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 162/333] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:39 2022] GET /identity/v3/users/3efa2868a5b641a4869a01194312b529 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:39.843858 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf3167-d33f-4e5d-a473-ee02a721aea1 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:39.980566 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf3167-d33f-4e5d-a473-ee02a721aea1 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:40.018869 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b76db-c145-4054-9102-27f6aa7d21e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:40.019483 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b76db-c145-4054-9102-27f6aa7d21e6 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:40.019760 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b76db-c145-4054-9102-27f6aa7d21e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:40.020032 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b76db-c145-4054-9102-27f6aa7d21e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:40.020810 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b76db-c145-4054-9102-27f6aa7d21e6 None admin] RBAC: Adding query filter params (role.id=934ee0e8483c496cb8813eccbe0ebf55, user.id=3efa2868a5b641a4869a01194312b529, scope.system=all) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:40.021118 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b76db-c145-4054-9102-27f6aa7d21e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:40.022617 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b76db-c145-4054-9102-27f6aa7d21e6 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:40.034847 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 172/334] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Sep 14 01:53:39 2022] GET /identity/v3/role_assignments?user.id=3efa2868a5b641a4869a01194312b529&scope.system=all&role.id=934ee0e8483c496cb8813eccbe0ebf55 => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:41.927851 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dd110-826c-45e2-a639-f696b76a4c74 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:41.928872 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dd110-826c-45e2-a639-f696b76a4c74 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:41.929379 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dd110-826c-45e2-a639-f696b76a4c74 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:41.931193 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 163/335] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:41.936861 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b3fa2-6678-42ad-9db3-164854eb1ab4 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:41.937659 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b3fa2-6678-42ad-9db3-164854eb1ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:41.938170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b3fa2-6678-42ad-9db3-164854eb1ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:41.967021 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-ef3b3fa2-6678-42ad-9db3-164854eb1ab4 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:41.968557 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-ef3b3fa2-6678-42ad-9db3-164854eb1ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:41.980732 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:42.056827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 173/336] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:53:42.318054 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c69858-84ce-41b4-9fb8-539f86d02b3a None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:42.318241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c69858-84ce-41b4-9fb8-539f86d02b3a None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:42.318523 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c69858-84ce-41b4-9fb8-539f86d02b3a None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:42.320150 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 164/337] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:42.325872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f920e-311f-4c2b-91eb-669d68379e68 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:42.326210 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f920e-311f-4c2b-91eb-669d68379e68 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:42.326484 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f920e-311f-4c2b-91eb-669d68379e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:42.350566 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-389f920e-311f-4c2b-91eb-669d68379e68 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:42.351738 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-389f920e-311f-4c2b-91eb-669d68379e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:42.444874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 174/338] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:42.626311 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c69858-84ce-41b4-9fb8-539f86d02b3a None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:42.711041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:42.762860 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c69858-84ce-41b4-9fb8-539f86d02b3a None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:42.781820 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f920e-311f-4c2b-91eb-669d68379e68 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:42.805322 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254ec79-0c9e-4701-87de-61d55070a20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:42.805997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254ec79-0c9e-4701-87de-61d55070a20a None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:42.806285 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254ec79-0c9e-4701-87de-61d55070a20a None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:42.806556 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254ec79-0c9e-4701-87de-61d55070a20a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:42.823989 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254ec79-0c9e-4701-87de-61d55070a20a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:42.829474 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-6254ec79-0c9e-4701-87de-61d55070a20a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:42.829474 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:42.829474 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:42.829474 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:42.829474 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:42.829474 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:42.829474 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:42.829474 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:42.829474 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:42.829474 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:42.829474 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:42.829474 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:42.829474 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:42.829474 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:42.829474 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:42.829474 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:42.830460 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:42.830460 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:42.830460 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:42.830460 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:42.830460 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:42.830460 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:42.830460 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:42.830460 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:42.830460 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:42.830460 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:42.830460 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:42.830460 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:42.830460 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:42.830460 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:42.830460 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:42.830460 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:42.831724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:42.831724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:42.831724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:42.831724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:42.831724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:42.831724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:42.831724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:42.831724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:42.831724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:42.831724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:42.831724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:42.831724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:42.831724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 165/339] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:53:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:42.839822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254ec79-0c9e-4701-87de-61d55070a20a None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:42.968759 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:43.002563 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f920e-311f-4c2b-91eb-669d68379e68 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:43.003786 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254ec79-0c9e-4701-87de-61d55070a20a None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:43.036270 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:43.048501 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e6030-9874-4ffc-a256-aada82235095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:43.049625 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e6030-9874-4ffc-a256-aada82235095 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:43.050115 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e6030-9874-4ffc-a256-aada82235095 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:43.050501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65bbef-ebc1-4545-b8e8-22492f73cc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:43.050723 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e6030-9874-4ffc-a256-aada82235095 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:43.051418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65bbef-ebc1-4545-b8e8-22492f73cc1e None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:43.051749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65bbef-ebc1-4545-b8e8-22492f73cc1e None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:43.052240 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65bbef-ebc1-4545-b8e8-22492f73cc1e None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:43.053389 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65bbef-ebc1-4545-b8e8-22492f73cc1e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:43.054311 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65bbef-ebc1-4545-b8e8-22492f73cc1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:43.055924 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65bbef-ebc1-4545-b8e8-22492f73cc1e None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:43.063328 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e6030-9874-4ffc-a256-aada82235095 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:43.066303 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 166/340] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:53:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:43.068273 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32986ef7-8c95-4785-a4c0-0c3b17c03e9d None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:43.068775 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32986ef7-8c95-4785-a4c0-0c3b17c03e9d None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:43.069260 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32986ef7-8c95-4785-a4c0-0c3b17c03e9d None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:43.070319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-2c4e6030-9874-4ffc-a256-aada82235095 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:43.070319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:43.070319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:43.070319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:43.070319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:43.070319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:43.070319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:43.070319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:43.070319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:43.070319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:43.070319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:43.070319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:43.070319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:43.070319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:43.070319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:43.070319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:43.071860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:43.071860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:43.071860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:43.071860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:43.071860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:43.071860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:43.071860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:43.071860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:43.071860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:43.071860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:43.071860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:43.071860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:43.071860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:43.071860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:43.071860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:43.071860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:43.073057 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:43.073057 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:43.073057 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:43.073057 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:43.073057 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:43.073057 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:43.073057 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:43.073057 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:43.073057 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:43.073057 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:43.073057 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:43.073057 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:43.073718 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 167/341] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:43.073718 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bca023-83a6-4627-9625-cba3451d30e3 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:43.073718 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bca023-83a6-4627-9625-cba3451d30e3 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:43.073718 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bca023-83a6-4627-9625-cba3451d30e3 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:43.078844 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 168/343] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:43.079033 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 175/343] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:43.079918 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bca023-83a6-4627-9625-cba3451d30e3 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:43.084377 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9dbe7b-8720-4df6-b010-527c2f975b24 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:43.084377 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9dbe7b-8720-4df6-b010-527c2f975b24 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:43.084377 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9dbe7b-8720-4df6-b010-527c2f975b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:43.104662 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-5b9dbe7b-8720-4df6-b010-527c2f975b24 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:43.105947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-5b9dbe7b-8720-4df6-b010-527c2f975b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:43.191963 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 176/344] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:43.193361 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d958f1-58a7-4064-82aa-48e96b431a86 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:43.193875 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d958f1-58a7-4064-82aa-48e96b431a86 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:43.202008 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d958f1-58a7-4064-82aa-48e96b431a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:43.207564 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bca023-83a6-4627-9625-cba3451d30e3 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:43.228860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-80d958f1-58a7-4064-82aa-48e96b431a86 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:43.230478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-80d958f1-58a7-4064-82aa-48e96b431a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:43.247701 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28570141-39f2-4245-9015-2a4b1f4b83da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:43.248134 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28570141-39f2-4245-9015-2a4b1f4b83da None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:43.248416 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28570141-39f2-4245-9015-2a4b1f4b83da None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:43.248686 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28570141-39f2-4245-9015-2a4b1f4b83da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:43.312756 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:43.316086 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:43.323689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 177/345] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:43.324824 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d958f1-58a7-4064-82aa-48e96b431a86 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:43.345490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28570141-39f2-4245-9015-2a4b1f4b83da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:43.353159 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28570141-39f2-4245-9015-2a4b1f4b83da None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:43.475820 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 169/346] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:43.477764 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f927358-f226-4b34-8f43-bc0267893a9a None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:43.478716 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f927358-f226-4b34-8f43-bc0267893a9a None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:43.479255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f927358-f226-4b34-8f43-bc0267893a9a None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:43.480734 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 170/347] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:43.483235 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f927358-f226-4b34-8f43-bc0267893a9a None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:43.531420 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d958f1-58a7-4064-82aa-48e96b431a86 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:43.569838 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d27d4-7cba-4a22-bd4e-895e138b93a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:43.570840 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d27d4-7cba-4a22-bd4e-895e138b93a0 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:43.571383 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d27d4-7cba-4a22-bd4e-895e138b93a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:43.571900 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d27d4-7cba-4a22-bd4e-895e138b93a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:43.572797 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d27d4-7cba-4a22-bd4e-895e138b93a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:43.573293 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d27d4-7cba-4a22-bd4e-895e138b93a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:43.574918 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d27d4-7cba-4a22-bd4e-895e138b93a0 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:43.584804 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 178/348] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:43.586478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a7cda-5441-4a9e-af53-fd511b86fb5b None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:43.586970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a7cda-5441-4a9e-af53-fd511b86fb5b None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:43.587505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a7cda-5441-4a9e-af53-fd511b86fb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:43.607668 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-4e9a7cda-5441-4a9e-af53-fd511b86fb5b None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:43.609201 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-4e9a7cda-5441-4a9e-af53-fd511b86fb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:43.668682 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:43.683110 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:43.685223 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 179/349] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:43.686311 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a7cda-5441-4a9e-af53-fd511b86fb5b None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:43.704945 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f927358-f226-4b34-8f43-bc0267893a9a None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:43.755547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201f7df-48ee-43b7-b29e-16a209524588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:43.755808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201f7df-48ee-43b7-b29e-16a209524588 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:43.755997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201f7df-48ee-43b7-b29e-16a209524588 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:43.756276 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201f7df-48ee-43b7-b29e-16a209524588 None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:43.757959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201f7df-48ee-43b7-b29e-16a209524588 None admin] RBAC: Authorizing `identity:get_project(project_id=f8542c9d54464124a730d0ce2363e406)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:43.759725 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201f7df-48ee-43b7-b29e-16a209524588 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:43.762617 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 171/350] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:43 2022] GET /identity/v3/projects/f8542c9d54464124a730d0ce2363e406 => generated 298 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:43.764102 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201f7df-48ee-43b7-b29e-16a209524588 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:43.819439 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a7cda-5441-4a9e-af53-fd511b86fb5b None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:43.858439 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db2373-2232-42d5-bd75-f9557f9d7f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:43.858932 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db2373-2232-42d5-bd75-f9557f9d7f55 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:43.859218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db2373-2232-42d5-bd75-f9557f9d7f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:43.859492 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db2373-2232-42d5-bd75-f9557f9d7f55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:43.907698 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201f7df-48ee-43b7-b29e-16a209524588 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:43.919152 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:43.933214 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db2373-2232-42d5-bd75-f9557f9d7f55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:43.934748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db2373-2232-42d5-bd75-f9557f9d7f55 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:43.948072 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0929f-2ff4-40f1-b554-6fc5670d55ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:43.949022 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0929f-2ff4-40f1-b554-6fc5670d55ca None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:43.949559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0929f-2ff4-40f1-b554-6fc5670d55ca None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:43.950070 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0929f-2ff4-40f1-b554-6fc5670d55ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:43.963050 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0929f-2ff4-40f1-b554-6fc5670d55ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:43.968280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-93d0929f-2ff4-40f1-b554-6fc5670d55ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:43.968280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:43.968280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:43.968280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:43.968280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:43.968280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:43.968280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:43.968280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:43.968280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:43.968280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:43.968280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:43.968280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:43.968280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:43.968280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:43.968280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:43.968280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:43.969223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:43.969223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:43.969223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:43.969223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:43.969223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:43.969223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:43.969223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:43.969223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:43.969223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:43.969223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:43.969223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:43.969223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:43.969223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:43.969223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:43.969223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:43.969223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:43.970451 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:43.970451 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:43.970451 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:43.970451 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:43.970451 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:43.970451 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:43.970451 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:43.970451 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:43.970451 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:43.970451 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:43.970451 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:43.970451 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:43.972847 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 172/351] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:43.973905 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0929f-2ff4-40f1-b554-6fc5670d55ca None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:43.996924 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:44.058114 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 180/352] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:44.062367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db2373-2232-42d5-bd75-f9557f9d7f55 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:44.108525 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0929f-2ff4-40f1-b554-6fc5670d55ca None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:44.150662 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2835be20-13a1-4405-9f61-4c748ad672d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:44.151801 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835be20-13a1-4405-9f61-4c748ad672d6 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:44.152080 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835be20-13a1-4405-9f61-4c748ad672d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:44.152511 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835be20-13a1-4405-9f61-4c748ad672d6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:44.165317 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2835be20-13a1-4405-9f61-4c748ad672d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:44.171735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-2835be20-13a1-4405-9f61-4c748ad672d6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:53:44.171735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:44.171735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:44.171735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:44.171735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:44.171735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:44.171735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:44.171735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:44.171735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:44.171735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:44.171735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:44.171735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:44.171735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:44.171735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:44.171735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:44.171735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:44.172865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:44.172865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:44.172865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:44.172865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:44.172865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:44.172865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:44.172865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:44.172865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:44.172865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:44.172865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:44.172865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:44.172865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:44.172865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:44.172865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:44.172865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:44.172865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:44.173682 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:44.173682 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:44.173682 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:44.173682 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:44.173682 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:44.173682 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:44.173682 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:44.173682 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:44.173682 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:44.173682 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:44.173682 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:53:44.173682 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:44.174403 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 173/353] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:44.174989 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2835be20-13a1-4405-9f61-4c748ad672d6 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:44.186691 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:44.199519 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db2373-2232-42d5-bd75-f9557f9d7f55 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:44.245423 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969e5ba-390f-4e54-9e6f-719a76f445b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:44.246225 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969e5ba-390f-4e54-9e6f-719a76f445b5 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:44.246509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969e5ba-390f-4e54-9e6f-719a76f445b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:44.247752 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969e5ba-390f-4e54-9e6f-719a76f445b5 None admin] PATH_INFO: `/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:44.256074 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3969e5ba-390f-4e54-9e6f-719a76f445b5 None admin] RBAC: Authorizing `identity:get_group(group_id=2e14b4b0fee94205981b90d7ddedbc9e)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:44.258196 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3969e5ba-390f-4e54-9e6f-719a76f445b5 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:44.260967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 181/354] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 14 01:53:44 2022] GET /identity/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:44.262048 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969e5ba-390f-4e54-9e6f-719a76f445b5 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:44.287394 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:44.310658 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2835be20-13a1-4405-9f61-4c748ad672d6 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:44.348561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbed51-6b24-4cdd-8032-f4bde53f2b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:44.349374 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbed51-6b24-4cdd-8032-f4bde53f2b70 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:44.349722 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbed51-6b24-4cdd-8032-f4bde53f2b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:44.350070 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbed51-6b24-4cdd-8032-f4bde53f2b70 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:44.360799 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cbed51-6b24-4cdd-8032-f4bde53f2b70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:44.366263 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-02cbed51-6b24-4cdd-8032-f4bde53f2b70 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:44.366263 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:44.366263 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:44.366263 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:44.366263 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:44.366263 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:44.366263 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:44.366263 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:44.366263 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:44.366263 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:44.366263 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:44.366263 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:44.366263 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:44.366263 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:44.366263 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:44.366263 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:44.367166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:44.367166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:44.367166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:44.367166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:44.367166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:44.367166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:44.367166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:44.367166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:44.367166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:44.367166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:44.367166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:44.367166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:44.367166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:44.367166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:44.367166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:44.367166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:44.367984 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:44.367984 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:44.367984 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:44.367984 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:44.367984 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:44.367984 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:44.367984 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:44.367984 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:44.367984 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:44.367984 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:44.367984 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:44.367984 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:44.367984 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 174/355] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:53:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:44.368692 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbed51-6b24-4cdd-8032-f4bde53f2b70 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:44.402503 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969e5ba-390f-4e54-9e6f-719a76f445b5 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:44.443038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fcc16f-03cf-4b80-84c7-e0d96aeb7cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:44.443657 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcc16f-03cf-4b80-84c7-e0d96aeb7cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:44.443935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcc16f-03cf-4b80-84c7-e0d96aeb7cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:44.444210 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcc16f-03cf-4b80-84c7-e0d96aeb7cb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:44.447257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fcc16f-03cf-4b80-84c7-e0d96aeb7cb6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:44.447257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fcc16f-03cf-4b80-84c7-e0d96aeb7cb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:44.447257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fcc16f-03cf-4b80-84c7-e0d96aeb7cb6 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:44.456193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 182/356] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:44.457045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fcc16f-03cf-4b80-84c7-e0d96aeb7cb6 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:44.476153 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:44.502189 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbed51-6b24-4cdd-8032-f4bde53f2b70 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:44.552456 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0d5bb-f325-4a81-a7f7-baf6ff40f41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:44.553373 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0d5bb-f325-4a81-a7f7-baf6ff40f41c None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:44.553650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0d5bb-f325-4a81-a7f7-baf6ff40f41c None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:44.553921 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0d5bb-f325-4a81-a7f7-baf6ff40f41c None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:44.555108 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0d5bb-f325-4a81-a7f7-baf6ff40f41c None admin] RBAC: Authorizing `identity:get_project(project_id=f8542c9d54464124a730d0ce2363e406)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:44.556552 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0d5bb-f325-4a81-a7f7-baf6ff40f41c None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:44.558861 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 175/357] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:44 2022] GET /identity/v3/projects/f8542c9d54464124a730d0ce2363e406 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:44.559999 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0d5bb-f325-4a81-a7f7-baf6ff40f41c None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:44.566350 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:44.586695 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fcc16f-03cf-4b80-84c7-e0d96aeb7cb6 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:44.627106 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5282f92-c56c-48ae-be3a-ca43f388d3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:44.627834 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5282f92-c56c-48ae-be3a-ca43f388d3d8 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:44.627986 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5282f92-c56c-48ae-be3a-ca43f388d3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:44.628240 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5282f92-c56c-48ae-be3a-ca43f388d3d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:44.630291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5282f92-c56c-48ae-be3a-ca43f388d3d8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:44.630291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5282f92-c56c-48ae-be3a-ca43f388d3d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:44.632046 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5282f92-c56c-48ae-be3a-ca43f388d3d8 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:44.640625 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 183/358] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:44.641723 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5282f92-c56c-48ae-be3a-ca43f388d3d8 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:44.678947 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0d5bb-f325-4a81-a7f7-baf6ff40f41c None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:44.720269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f877ef-890d-4632-9702-57720098cbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:44.720879 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f877ef-890d-4632-9702-57720098cbe2 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:44.721154 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f877ef-890d-4632-9702-57720098cbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:44.721420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f877ef-890d-4632-9702-57720098cbe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:44.722274 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f877ef-890d-4632-9702-57720098cbe2 None admin] RBAC: Adding query filter params (group.id=2e14b4b0fee94205981b90d7ddedbc9e, role.id=e842ac50986946a4971a51a06a39e16b, scope.project.id=f8542c9d54464124a730d0ce2363e406) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:44.722576 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f877ef-890d-4632-9702-57720098cbe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:44.724204 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f877ef-890d-4632-9702-57720098cbe2 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:44.730378 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 176/359] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Sep 14 01:53:44 2022] GET /identity/v3/role_assignments?group.id=2e14b4b0fee94205981b90d7ddedbc9e&scope.project.id=f8542c9d54464124a730d0ce2363e406&role.id=e842ac50986946a4971a51a06a39e16b => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:44.731259 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f877ef-890d-4632-9702-57720098cbe2 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:44.734887 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:44.768139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5282f92-c56c-48ae-be3a-ca43f388d3d8 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:44.818511 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603de03-46f5-47c1-b27c-ccc3de4443fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:44.819437 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603de03-46f5-47c1-b27c-ccc3de4443fe None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:44.820730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603de03-46f5-47c1-b27c-ccc3de4443fe None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:44.820730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603de03-46f5-47c1-b27c-ccc3de4443fe None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:44.822490 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603de03-46f5-47c1-b27c-ccc3de4443fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:44.822490 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603de03-46f5-47c1-b27c-ccc3de4443fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:44.823579 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603de03-46f5-47c1-b27c-ccc3de4443fe None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:44.828890 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:44.837768 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 184/360] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:53:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:44.839043 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603de03-46f5-47c1-b27c-ccc3de4443fe None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:44.878870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f877ef-890d-4632-9702-57720098cbe2 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:44.928252 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e630e-9a1c-416e-80df-e5648d62baca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:44.928876 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e630e-9a1c-416e-80df-e5648d62baca None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:44.929154 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e630e-9a1c-416e-80df-e5648d62baca None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:44.929426 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e630e-9a1c-416e-80df-e5648d62baca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:44.970054 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603de03-46f5-47c1-b27c-ccc3de4443fe None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:45.013409 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4cc0a5-87ea-4696-9022-4e85c8312475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:45.017034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4cc0a5-87ea-4696-9022-4e85c8312475 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:45.017034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4cc0a5-87ea-4696-9022-4e85c8312475 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:45.017034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4cc0a5-87ea-4696-9022-4e85c8312475 None admin] PATH_INFO: `/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:45.018852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4cc0a5-87ea-4696-9022-4e85c8312475 None admin] RBAC: Authorizing `identity:get_group(group_id=2e14b4b0fee94205981b90d7ddedbc9e)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:45.020599 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4cc0a5-87ea-4696-9022-4e85c8312475 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:45.020943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e630e-9a1c-416e-80df-e5648d62baca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:45.021840 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 185/361] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 14 01:53:44 2022] GET /identity/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:45.024632 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b8175-0f43-4ede-bdca-5f5b170996fe None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:45.024632 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b8175-0f43-4ede-bdca-5f5b170996fe None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:45.024632 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b8175-0f43-4ede-bdca-5f5b170996fe None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:45.025822 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 186/362] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:45.027215 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b8175-0f43-4ede-bdca-5f5b170996fe None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:45.028087 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e630e-9a1c-416e-80df-e5648d62baca None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:45.037375 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:45.121377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:45.164915 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 177/363] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:45.165787 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e630e-9a1c-416e-80df-e5648d62baca None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:45.171053 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b8175-0f43-4ede-bdca-5f5b170996fe None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:45.213160 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaaa966-eae9-4787-a58b-cff83803ef1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:45.213801 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaaa966-eae9-4787-a58b-cff83803ef1c None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:45.214130 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaaa966-eae9-4787-a58b-cff83803ef1c None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:45.214405 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaaa966-eae9-4787-a58b-cff83803ef1c None admin] PATH_INFO: `/v3/users/9442e5a46ace4cc3a925f434519c0ae2` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:45.215996 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaaa966-eae9-4787-a58b-cff83803ef1c None admin] RBAC: Authorizing `identity:get_user(user_id=9442e5a46ace4cc3a925f434519c0ae2)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:45.217464 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaaa966-eae9-4787-a58b-cff83803ef1c None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:45.220081 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 187/364] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:53:45 2022] GET /identity/v3/users/9442e5a46ace4cc3a925f434519c0ae2 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:45.221224 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaaa966-eae9-4787-a58b-cff83803ef1c None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:45.300227 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e630e-9a1c-416e-80df-e5648d62baca None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:45.314286 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:45.338947 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151164b-292f-4e14-ae3a-e8d759e7b65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:45.339621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151164b-292f-4e14-ae3a-e8d759e7b65b None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:45.339844 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151164b-292f-4e14-ae3a-e8d759e7b65b None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:45.340368 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151164b-292f-4e14-ae3a-e8d759e7b65b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:45.344916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaaa966-eae9-4787-a58b-cff83803ef1c None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:45.388903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:45.398985 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3cc22-b40f-44a1-9753-dd9f1be35e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:45.400168 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3cc22-b40f-44a1-9753-dd9f1be35e40 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:45.401355 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3cc22-b40f-44a1-9753-dd9f1be35e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:45.401916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3cc22-b40f-44a1-9753-dd9f1be35e40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:45.402861 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f3cc22-b40f-44a1-9753-dd9f1be35e40 None admin] RBAC: Adding query filter params (group.id=2e14b4b0fee94205981b90d7ddedbc9e, role.id=934ee0e8483c496cb8813eccbe0ebf55, scope.project.id=f8542c9d54464124a730d0ce2363e406) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:45.403171 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f3cc22-b40f-44a1-9753-dd9f1be35e40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:45.404615 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f3cc22-b40f-44a1-9753-dd9f1be35e40 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:45.411960 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 188/365] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Sep 14 01:53:45 2022] GET /identity/v3/role_assignments?group.id=2e14b4b0fee94205981b90d7ddedbc9e&scope.project.id=f8542c9d54464124a730d0ce2363e406&role.id=934ee0e8483c496cb8813eccbe0ebf55 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:45.412953 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d73bbe-af19-49d0-92ee-73cde0cd7ce4 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:45.413236 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d73bbe-af19-49d0-92ee-73cde0cd7ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:45.413506 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d73bbe-af19-49d0-92ee-73cde0cd7ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:45.433891 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4151164b-292f-4e14-ae3a-e8d759e7b65b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:45.434827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-17d73bbe-af19-49d0-92ee-73cde0cd7ce4 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:45.436030 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-17d73bbe-af19-49d0-92ee-73cde0cd7ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:45.436320 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4151164b-292f-4e14-ae3a-e8d759e7b65b None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:45.523634 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 189/366] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:45.524534 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d73bbe-af19-49d0-92ee-73cde0cd7ce4 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:45.562008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 178/367] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:45.569310 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151164b-292f-4e14-ae3a-e8d759e7b65b None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:45.605136 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:45.642977 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:45.644193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d73bbe-af19-49d0-92ee-73cde0cd7ce4 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:45.681553 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151164b-292f-4e14-ae3a-e8d759e7b65b None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:45.681744 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a200db-a592-42da-9373-ab1838610a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:45.682261 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a200db-a592-42da-9373-ab1838610a67 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:45.682537 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a200db-a592-42da-9373-ab1838610a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:45.682804 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a200db-a592-42da-9373-ab1838610a67 None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:45.683924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a200db-a592-42da-9373-ab1838610a67 None admin] RBAC: Authorizing `identity:get_project(project_id=60ea1e9a961b4e3b99b9fe44ac610808)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:45.685840 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a200db-a592-42da-9373-ab1838610a67 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:45.687005 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 190/368] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:45 2022] GET /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:45.687760 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a200db-a592-42da-9373-ab1838610a67 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:45.724946 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71178d-734a-4b5d-a462-de1cda25ab7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:45.725842 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71178d-734a-4b5d-a462-de1cda25ab7b None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:45.726282 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71178d-734a-4b5d-a462-de1cda25ab7b None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:45.726715 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71178d-734a-4b5d-a462-de1cda25ab7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:45.727578 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c71178d-734a-4b5d-a462-de1cda25ab7b None admin] RBAC: Adding query filter params (role.id=e5ce565bb4ca4bad8c14b5e76d2ae8ea, user.id=9442e5a46ace4cc3a925f434519c0ae2, scope.system=all) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:45.728042 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c71178d-734a-4b5d-a462-de1cda25ab7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:45.729632 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c71178d-734a-4b5d-a462-de1cda25ab7b None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:45.743307 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 179/369] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Sep 14 01:53:45 2022] GET /identity/v3/role_assignments?user.id=9442e5a46ace4cc3a925f434519c0ae2&scope.system=all&role.id=e5ce565bb4ca4bad8c14b5e76d2ae8ea => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:45.744338 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71178d-734a-4b5d-a462-de1cda25ab7b None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:45.813674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a200db-a592-42da-9373-ab1838610a67 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:45.860250 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71178d-734a-4b5d-a462-de1cda25ab7b None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:45.870771 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db034864-918e-4dbf-a28f-a1579f99fcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:45.871424 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db034864-918e-4dbf-a28f-a1579f99fcd0 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:45.871697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db034864-918e-4dbf-a28f-a1579f99fcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:45.871962 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db034864-918e-4dbf-a28f-a1579f99fcd0 None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:45.873172 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db034864-918e-4dbf-a28f-a1579f99fcd0 None admin] RBAC: Authorizing `identity:get_project(project_id=60ea1e9a961b4e3b99b9fe44ac610808)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:45.874812 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db034864-918e-4dbf-a28f-a1579f99fcd0 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:45.876239 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 191/370] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:45 2022] GET /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:45.882355 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db034864-918e-4dbf-a28f-a1579f99fcd0 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:45.889444 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:45.890449 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:45.910887 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b0bff-c6c4-4e39-be44-7f847905ef7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:45.911463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b0bff-c6c4-4e39-be44-7f847905ef7d None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:45.911742 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b0bff-c6c4-4e39-be44-7f847905ef7d None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:45.912132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b0bff-c6c4-4e39-be44-7f847905ef7d None admin] PATH_INFO: `/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:45.918241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b0bff-c6c4-4e39-be44-7f847905ef7d None admin] RBAC: Authorizing `identity:get_group(group_id=2e14b4b0fee94205981b90d7ddedbc9e)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:45.919211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b0bff-c6c4-4e39-be44-7f847905ef7d None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:45.921287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 180/371] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 14 01:53:45 2022] GET /identity/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:45.922114 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b0bff-c6c4-4e39-be44-7f847905ef7d None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:46.048059 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db034864-918e-4dbf-a28f-a1579f99fcd0 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:46.056710 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b0bff-c6c4-4e39-be44-7f847905ef7d None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:46.096408 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098a0fb-8991-4996-a020-a0fc3a966e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:46.097042 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098a0fb-8991-4996-a020-a0fc3a966e99 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:46.097642 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098a0fb-8991-4996-a020-a0fc3a966e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:46.098712 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098a0fb-8991-4996-a020-a0fc3a966e99 None admin] PATH_INFO: `/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:46.099825 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098a0fb-8991-4996-a020-a0fc3a966e99 None admin] RBAC: Authorizing `identity:get_group(group_id=2e14b4b0fee94205981b90d7ddedbc9e)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:46.101257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098a0fb-8991-4996-a020-a0fc3a966e99 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:46.102710 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26707250-2974-4808-bda8-6c91f6a09ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:46.103266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 192/372] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 14 01:53:45 2022] GET /identity/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:46.103375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26707250-2974-4808-bda8-6c91f6a09ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:46.103611 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26707250-2974-4808-bda8-6c91f6a09ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:46.103881 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26707250-2974-4808-bda8-6c91f6a09ba8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:46.104043 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098a0fb-8991-4996-a020-a0fc3a966e99 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:46.121177 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26707250-2974-4808-bda8-6c91f6a09ba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:46.129677 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-26707250-2974-4808-bda8-6c91f6a09ba8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:53:46.129677 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:46.129677 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:46.129677 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:46.129677 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:46.129677 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:46.129677 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:46.129677 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:46.129677 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:46.129677 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:46.129677 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:46.129677 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:46.129677 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:46.129677 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:46.129677 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:46.129677 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:46.131113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:46.131113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:46.131113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:46.131113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:46.131113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:46.131113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:46.131113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:46.131113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:46.131113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:46.131113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:46.131113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:46.131113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:46.131113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:46.131113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:46.131113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:46.131113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:46.132220 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:46.132220 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:46.132220 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:46.132220 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:46.132220 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:46.132220 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:46.132220 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:46.132220 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:46.132220 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:46.132220 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:46.132220 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:53:46.132220 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:46.132220 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 181/373] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:53:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:46.132899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26707250-2974-4808-bda8-6c91f6a09ba8 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:46.205935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:46.212128 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:46.253134 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098a0fb-8991-4996-a020-a0fc3a966e99 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:46.281200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26707250-2974-4808-bda8-6c91f6a09ba8 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:46.302698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd259a-f32f-4afa-812f-f41983019fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:46.304069 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd259a-f32f-4afa-812f-f41983019fef None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:46.304521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd259a-f32f-4afa-812f-f41983019fef None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:46.305176 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd259a-f32f-4afa-812f-f41983019fef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:46.306321 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd259a-f32f-4afa-812f-f41983019fef None admin] RBAC: Adding query filter params (group.id=2e14b4b0fee94205981b90d7ddedbc9e, role.id=934ee0e8483c496cb8813eccbe0ebf55, scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:46.306647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd259a-f32f-4afa-812f-f41983019fef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:46.308109 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd259a-f32f-4afa-812f-f41983019fef None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:46.317308 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 193/374] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Sep 14 01:53:46 2022] GET /identity/v3/role_assignments?group.id=2e14b4b0fee94205981b90d7ddedbc9e&scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808&role.id=934ee0e8483c496cb8813eccbe0ebf55 => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:46.317928 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd259a-f32f-4afa-812f-f41983019fef None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:46.350792 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c43ae0-a391-41d7-bf04-10304c428f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:46.351592 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c43ae0-a391-41d7-bf04-10304c428f5f None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:46.351877 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c43ae0-a391-41d7-bf04-10304c428f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:46.352152 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c43ae0-a391-41d7-bf04-10304c428f5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:46.352899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c43ae0-a391-41d7-bf04-10304c428f5f None admin] RBAC: Adding query filter params (group.id=2e14b4b0fee94205981b90d7ddedbc9e, role.id=e842ac50986946a4971a51a06a39e16b, scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:46.353207 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c43ae0-a391-41d7-bf04-10304c428f5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:46.354701 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c43ae0-a391-41d7-bf04-10304c428f5f None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:46.368358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 182/375] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Sep 14 01:53:46 2022] GET /identity/v3/role_assignments?group.id=2e14b4b0fee94205981b90d7ddedbc9e&scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808&role.id=e842ac50986946a4971a51a06a39e16b => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:46.447312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd259a-f32f-4afa-812f-f41983019fef None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:46.489456 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44961e-0bf0-475c-9c09-f494cb83c8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:46.490490 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44961e-0bf0-475c-9c09-f494cb83c8c6 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:46.490970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44961e-0bf0-475c-9c09-f494cb83c8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:46.491480 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44961e-0bf0-475c-9c09-f494cb83c8c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:46.492396 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44961e-0bf0-475c-9c09-f494cb83c8c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:46.492907 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44961e-0bf0-475c-9c09-f494cb83c8c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:46.494566 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44961e-0bf0-475c-9c09-f494cb83c8c6 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:46.497673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:46.509063 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 194/376] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:46.514688 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c43ae0-a391-41d7-bf04-10304c428f5f None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:46.651380 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c43ae0-a391-41d7-bf04-10304c428f5f None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:46.671869 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:46.691470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413bbb0f-3776-4edc-a255-d359154798f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:46.692140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413bbb0f-3776-4edc-a255-d359154798f9 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:46.692416 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413bbb0f-3776-4edc-a255-d359154798f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:46.692685 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413bbb0f-3776-4edc-a255-d359154798f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:46.766884 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413bbb0f-3776-4edc-a255-d359154798f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:46.768301 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413bbb0f-3776-4edc-a255-d359154798f9 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:46.878759 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 183/377] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:46.882943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44961e-0bf0-475c-9c09-f494cb83c8c6 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:47.019840 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44961e-0bf0-475c-9c09-f494cb83c8c6 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:47.058638 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a62318-0fcd-4f03-93cb-01a71358c174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:47.059274 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a62318-0fcd-4f03-93cb-01a71358c174 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:47.059554 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a62318-0fcd-4f03-93cb-01a71358c174 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:47.059822 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a62318-0fcd-4f03-93cb-01a71358c174 None admin] PATH_INFO: `/v3/projects/2ca72c26a2124532970b02d36e64c3f9` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:47.060966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a62318-0fcd-4f03-93cb-01a71358c174 None admin] RBAC: Authorizing `identity:get_project(project_id=2ca72c26a2124532970b02d36e64c3f9)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:47.062432 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a62318-0fcd-4f03-93cb-01a71358c174 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:47.064343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 195/378] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:46 2022] GET /identity/v3/projects/2ca72c26a2124532970b02d36e64c3f9 => generated 344 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:47.070377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413bbb0f-3776-4edc-a255-d359154798f9 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:47.092198 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:47.174861 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413bbb0f-3776-4edc-a255-d359154798f9 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:47.210517 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9132b1a-876f-4c28-9397-b6ec257bc35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:47.211161 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9132b1a-876f-4c28-9397-b6ec257bc35b None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:47.211444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9132b1a-876f-4c28-9397-b6ec257bc35b None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:47.211720 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9132b1a-876f-4c28-9397-b6ec257bc35b None admin] PATH_INFO: `/v3/groups/c4e92f5e40544357b755938ebc5b8a05` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:47.217726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9132b1a-876f-4c28-9397-b6ec257bc35b None admin] RBAC: Authorizing `identity:get_group(group_id=c4e92f5e40544357b755938ebc5b8a05)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:47.219240 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9132b1a-876f-4c28-9397-b6ec257bc35b None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:47.221161 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 184/379] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 14 01:53:47 2022] GET /identity/v3/groups/c4e92f5e40544357b755938ebc5b8a05 => generated 229 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:47.227026 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a62318-0fcd-4f03-93cb-01a71358c174 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:47.294757 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:47.336308 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a62318-0fcd-4f03-93cb-01a71358c174 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:47.377909 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495b59a-0138-4e7b-b0d9-52f7ad4cfe24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:47.378546 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495b59a-0138-4e7b-b0d9-52f7ad4cfe24 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:47.378823 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495b59a-0138-4e7b-b0d9-52f7ad4cfe24 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:47.379150 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495b59a-0138-4e7b-b0d9-52f7ad4cfe24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:47.379875 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495b59a-0138-4e7b-b0d9-52f7ad4cfe24 None admin] RBAC: Adding query filter params (group.id=c4e92f5e40544357b755938ebc5b8a05, role.id=aa420c2fb1ab4bb3b05b004a4d89779a, scope.project.id=2ca72c26a2124532970b02d36e64c3f9) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:47.380176 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495b59a-0138-4e7b-b0d9-52f7ad4cfe24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:47.381862 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495b59a-0138-4e7b-b0d9-52f7ad4cfe24 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:47.390201 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 196/380] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Sep 14 01:53:47 2022] GET /identity/v3/role_assignments?group.id=c4e92f5e40544357b755938ebc5b8a05&scope.project.id=2ca72c26a2124532970b02d36e64c3f9&role.id=aa420c2fb1ab4bb3b05b004a4d89779a => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:47.677420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c3afc-5fcc-4816-97d7-a93eb13ed03d None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:47.677799 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c3afc-5fcc-4816-97d7-a93eb13ed03d None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:47.678214 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c3afc-5fcc-4816-97d7-a93eb13ed03d None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:47.679911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 185/381] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:47.685243 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d91619-b11d-4865-88c3-0fde84beaed0 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:47.685533 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d91619-b11d-4865-88c3-0fde84beaed0 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:47.685810 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d91619-b11d-4865-88c3-0fde84beaed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:47.710474 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-09d91619-b11d-4865-88c3-0fde84beaed0 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:47.711659 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-09d91619-b11d-4865-88c3-0fde84beaed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:47.797717 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 197/382] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:48.099827 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c3afc-5fcc-4816-97d7-a93eb13ed03d None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:48.202542 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:48.207426 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc41a7e-4572-4511-aca6-59abf36732f5 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:48.207720 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc41a7e-4572-4511-aca6-59abf36732f5 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:48.207992 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc41a7e-4572-4511-aca6-59abf36732f5 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:48.209714 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 198/383] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:53:48.214889 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791f1ce-07e2-41bc-8ad3-6e0bd05186a4 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:48.215167 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791f1ce-07e2-41bc-8ad3-6e0bd05186a4 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:48.215437 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791f1ce-07e2-41bc-8ad3-6e0bd05186a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:48.231255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c3afc-5fcc-4816-97d7-a93eb13ed03d None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:48.239145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-b791f1ce-07e2-41bc-8ad3-6e0bd05186a4 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:48.240355 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-b791f1ce-07e2-41bc-8ad3-6e0bd05186a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:48.299612 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb87286-4374-4776-9f69-964efd787f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:48.300244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb87286-4374-4776-9f69-964efd787f65 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:48.300521 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb87286-4374-4776-9f69-964efd787f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:48.300789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb87286-4374-4776-9f69-964efd787f65 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:48.321513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb87286-4374-4776-9f69-964efd787f65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:48.326356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-fcb87286-4374-4776-9f69-964efd787f65 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:48.326356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:48.326356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:48.326356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:48.326356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:48.326356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:48.326356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:48.326356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:48.326356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:48.326356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:48.326356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:48.326356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:48.326356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:48.326356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:48.326356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:48.326356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:48.327222 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:48.327222 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:48.327222 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:48.327222 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:48.327222 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:48.327222 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:48.327222 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:48.327222 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:48.327222 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:48.327222 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:48.327222 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:48.327222 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:48.327222 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:48.327222 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:48.327222 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:48.327222 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:48.328041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:48.328041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:48.328041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:48.328041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:48.328041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:48.328041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:48.328041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:48.328041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:48.328041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:48.328041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:48.328041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:48.328041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:48.328041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 186/384] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:53:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:48.333373 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb87286-4374-4776-9f69-964efd787f65 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:48.358415 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:48.376115 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 199/385] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:53:48.483696 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb87286-4374-4776-9f69-964efd787f65 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:48.529357 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f1439-9739-473b-9e82-ae98bcfdfcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:48.530013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f1439-9739-473b-9e82-ae98bcfdfcb7 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:48.530269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f1439-9739-473b-9e82-ae98bcfdfcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:48.530534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f1439-9739-473b-9e82-ae98bcfdfcb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:48.531201 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f1439-9739-473b-9e82-ae98bcfdfcb7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:48.531496 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f1439-9739-473b-9e82-ae98bcfdfcb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:48.532854 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f1439-9739-473b-9e82-ae98bcfdfcb7 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:48.541448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 187/386] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:53:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:48.547092 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791f1ce-07e2-41bc-8ad3-6e0bd05186a4 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:48.678266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791f1ce-07e2-41bc-8ad3-6e0bd05186a4 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:48.722714 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadc6c1-710a-49fe-b92d-c0bcb6bd30d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:48.723385 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadc6c1-710a-49fe-b92d-c0bcb6bd30d0 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:48.723665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadc6c1-710a-49fe-b92d-c0bcb6bd30d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:48.723919 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadc6c1-710a-49fe-b92d-c0bcb6bd30d0 None admin] PATH_INFO: `/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:48.725022 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadc6c1-710a-49fe-b92d-c0bcb6bd30d0 None admin] RBAC: Authorizing `identity:get_group(group_id=2e14b4b0fee94205981b90d7ddedbc9e)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:48.726490 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadc6c1-710a-49fe-b92d-c0bcb6bd30d0 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:48.728394 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 200/387] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 14 01:53:48 2022] GET /identity/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:48.733552 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f1439-9739-473b-9e82-ae98bcfdfcb7 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:48.736440 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:48.815070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadc6c1-710a-49fe-b92d-c0bcb6bd30d0 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:48.877911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f1439-9739-473b-9e82-ae98bcfdfcb7 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:48.913213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3d7c9-75fb-4dfe-b0b2-e291c289f566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:48.913855 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3d7c9-75fb-4dfe-b0b2-e291c289f566 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:48.914176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3d7c9-75fb-4dfe-b0b2-e291c289f566 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:48.914449 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3d7c9-75fb-4dfe-b0b2-e291c289f566 None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:48.915575 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a3d7c9-75fb-4dfe-b0b2-e291c289f566 None admin] RBAC: Authorizing `identity:get_project(project_id=f8542c9d54464124a730d0ce2363e406)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:48.917009 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a3d7c9-75fb-4dfe-b0b2-e291c289f566 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:48.918959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 188/388] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:48 2022] GET /identity/v3/projects/f8542c9d54464124a730d0ce2363e406 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:48.925139 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3d7c9-75fb-4dfe-b0b2-e291c289f566 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:48.934391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:49.008594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadc6c1-710a-49fe-b92d-c0bcb6bd30d0 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:49.056437 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:49.079636 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5e24b-cc50-4495-8142-072177865c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:49.080269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5e24b-cc50-4495-8142-072177865c19 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:49.080543 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5e24b-cc50-4495-8142-072177865c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:49.082269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5e24b-cc50-4495-8142-072177865c19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:49.106065 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5e24b-cc50-4495-8142-072177865c19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:49.110957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-5be5e24b-cc50-4495-8142-072177865c19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:49.110957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:49.110957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:49.110957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:49.110957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:49.110957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:49.110957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:49.110957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:49.110957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:49.110957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:49.110957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:49.110957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:49.110957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:49.110957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:49.110957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:49.110957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:49.111860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:49.111860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:49.111860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:49.111860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:49.111860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:49.111860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:49.111860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:49.111860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:49.111860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:49.111860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:49.111860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:49.111860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:49.111860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:49.111860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:49.111860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:49.111860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:49.112663 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:49.112663 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:49.112663 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:49.112663 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:49.112663 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:49.112663 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:49.112663 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:49.112663 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:49.112663 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:49.112663 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:49.112663 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:49.112663 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:49.119174 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 201/389] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:49.119477 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3d7c9-75fb-4dfe-b0b2-e291c289f566 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:49.126237 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5e24b-cc50-4495-8142-072177865c19 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:49.154747 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6695a-a3bf-41c8-bd12-de7a772b6cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:49.155408 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6695a-a3bf-41c8-bd12-de7a772b6cad None admin] REQUEST_METHOD: `PUT` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:49.155682 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6695a-a3bf-41c8-bd12-de7a772b6cad None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:49.155948 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6695a-a3bf-41c8-bd12-de7a772b6cad None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406/groups/2e14b4b0fee94205981b90d7ddedbc9e/roles/e842ac50986946a4971a51a06a39e16b` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:49.158132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d6695a-a3bf-41c8-bd12-de7a772b6cad None admin] RBAC: Authorizing `identity:create_grant(project_id=f8542c9d54464124a730d0ce2363e406, group_id=2e14b4b0fee94205981b90d7ddedbc9e, role_id=e842ac50986946a4971a51a06a39e16b)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:49.159578 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d6695a-a3bf-41c8-bd12-de7a772b6cad None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:49.189595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 189/390] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Sep 14 01:53:48 2022] PUT /identity/v3/projects/f8542c9d54464124a730d0ce2363e406/groups/2e14b4b0fee94205981b90d7ddedbc9e/roles/e842ac50986946a4971a51a06a39e16b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:53:49.276724 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5e24b-cc50-4495-8142-072177865c19 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:49.308260 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:49.317915 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4d6e0-bd0c-4cb8-84a3-a5f460991e1f None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:49.318914 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4d6e0-bd0c-4cb8-84a3-a5f460991e1f None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:49.319621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4d6e0-bd0c-4cb8-84a3-a5f460991e1f None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:49.321277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 190/391] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:49.324320 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe62af-6045-4294-8632-ecf93ecd33ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:49.324921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe62af-6045-4294-8632-ecf93ecd33ec None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:49.325201 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe62af-6045-4294-8632-ecf93ecd33ec None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:49.325476 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe62af-6045-4294-8632-ecf93ecd33ec None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:49.326331 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe62af-6045-4294-8632-ecf93ecd33ec None admin] RBAC: Adding query filter params (name=member) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:49.326639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe62af-6045-4294-8632-ecf93ecd33ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:49.328046 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe62af-6045-4294-8632-ecf93ecd33ec None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:49.331658 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1d3bd-3163-42de-93cd-2dbf7b43954b None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:49.334750 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1d3bd-3163-42de-93cd-2dbf7b43954b None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:49.335140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1d3bd-3163-42de-93cd-2dbf7b43954b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:49.342018 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 202/392] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:49.343652 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe62af-6045-4294-8632-ecf93ecd33ec None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:49.372565 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-adf1d3bd-3163-42de-93cd-2dbf7b43954b None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:49.376195 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-adf1d3bd-3163-42de-93cd-2dbf7b43954b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:49.465377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 191/393] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:49.466362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94409c-5b42-488f-b276-a6a6f8fa2b84 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:49.466643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94409c-5b42-488f-b276-a6a6f8fa2b84 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:49.466906 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94409c-5b42-488f-b276-a6a6f8fa2b84 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:49.468133 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 192/394] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:49.473457 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e492227-88b0-4bfc-8c11-55be4259896f None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:49.473746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e492227-88b0-4bfc-8c11-55be4259896f None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:49.474044 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e492227-88b0-4bfc-8c11-55be4259896f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:49.498462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-4e492227-88b0-4bfc-8c11-55be4259896f None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:49.499950 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-4e492227-88b0-4bfc-8c11-55be4259896f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:49.504142 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe62af-6045-4294-8632-ecf93ecd33ec None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:49.566699 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:49.572209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 193/395] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:53:49.595608 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97766c8-290e-47a4-944c-c87dc336fbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:49.596265 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97766c8-290e-47a4-944c-c87dc336fbdf None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:49.596545 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97766c8-290e-47a4-944c-c87dc336fbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:49.596818 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97766c8-290e-47a4-944c-c87dc336fbdf None admin] PATH_INFO: `/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:49.602652 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97766c8-290e-47a4-944c-c87dc336fbdf None admin] RBAC: Authorizing `identity:get_group(group_id=2e14b4b0fee94205981b90d7ddedbc9e)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:49.604122 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97766c8-290e-47a4-944c-c87dc336fbdf None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:49.605719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 203/396] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 14 01:53:49 2022] GET /identity/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:49.611267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e492227-88b0-4bfc-8c11-55be4259896f None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:49.715678 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e492227-88b0-4bfc-8c11-55be4259896f None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:49.750786 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a6a93-1d2c-443b-acdf-3f6a6b6831c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:49.751761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a6a93-1d2c-443b-acdf-3f6a6b6831c3 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:49.752233 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a6a93-1d2c-443b-acdf-3f6a6b6831c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:49.752733 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a6a93-1d2c-443b-acdf-3f6a6b6831c3 None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:49.754651 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664a6a93-1d2c-443b-acdf-3f6a6b6831c3 None admin] RBAC: Authorizing `identity:get_project(project_id=f8542c9d54464124a730d0ce2363e406)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:49.756339 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664a6a93-1d2c-443b-acdf-3f6a6b6831c3 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:49.758721 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 194/397] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:49 2022] GET /identity/v3/projects/f8542c9d54464124a730d0ce2363e406 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:49.764128 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97766c8-290e-47a4-944c-c87dc336fbdf None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:49.846534 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:49.883115 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a6a93-1d2c-443b-acdf-3f6a6b6831c3 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:49.913298 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97766c8-290e-47a4-944c-c87dc336fbdf None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:49.946019 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:49.959419 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e944c-21d6-4741-bf49-7db27aaa22f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:49.961398 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e944c-21d6-4741-bf49-7db27aaa22f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:49.961891 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e944c-21d6-4741-bf49-7db27aaa22f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:49.962357 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e944c-21d6-4741-bf49-7db27aaa22f2 None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406/groups/2e14b4b0fee94205981b90d7ddedbc9e/roles/934ee0e8483c496cb8813eccbe0ebf55` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:49.964446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e944c-21d6-4741-bf49-7db27aaa22f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8542c9d54464124a730d0ce2363e406, group_id=2e14b4b0fee94205981b90d7ddedbc9e, role_id=934ee0e8483c496cb8813eccbe0ebf55)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:49.966653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e944c-21d6-4741-bf49-7db27aaa22f2 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:49.988438 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 204/398] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Sep 14 01:53:49 2022] PUT /identity/v3/projects/f8542c9d54464124a730d0ce2363e406/groups/2e14b4b0fee94205981b90d7ddedbc9e/roles/934ee0e8483c496cb8813eccbe0ebf55 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:53:49.993901 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e944c-21d6-4741-bf49-7db27aaa22f2 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:50.016834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a6a93-1d2c-443b-acdf-3f6a6b6831c3 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:50.089933 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af9629-2e6a-4331-8a42-68c6cc941873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:50.090600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af9629-2e6a-4331-8a42-68c6cc941873 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:50.090872 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af9629-2e6a-4331-8a42-68c6cc941873 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:50.091134 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af9629-2e6a-4331-8a42-68c6cc941873 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:50.114355 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af9629-2e6a-4331-8a42-68c6cc941873 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:50.120862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-43af9629-2e6a-4331-8a42-68c6cc941873 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:50.120862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:50.120862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:50.120862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:50.120862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:50.120862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:50.120862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:50.120862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:50.120862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:50.120862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:50.120862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:50.120862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:50.120862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:50.120862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:50.120862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:50.120862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:50.121870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:50.121870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:50.121870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:50.121870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:50.121870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:50.121870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:50.121870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:50.121870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:50.121870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:50.121870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:50.121870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:50.121870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:50.121870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:50.121870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:50.121870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:50.121870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:50.122735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:50.122735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:50.122735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:50.122735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:50.122735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:50.122735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:50.122735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:50.122735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:50.122735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:50.122735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:50.122735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:50.122735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:50.122735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 195/399] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:53:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:50.130759 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af9629-2e6a-4331-8a42-68c6cc941873 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:50.192520 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e944c-21d6-4741-bf49-7db27aaa22f2 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:50.205667 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:50.240908 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75eaacf-6445-4379-9528-ff3df2152fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:50.241752 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75eaacf-6445-4379-9528-ff3df2152fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:50.242060 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75eaacf-6445-4379-9528-ff3df2152fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:50.242328 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75eaacf-6445-4379-9528-ff3df2152fd8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:50.260073 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75eaacf-6445-4379-9528-ff3df2152fd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:50.267132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af9629-2e6a-4331-8a42-68c6cc941873 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:50.267995 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-b75eaacf-6445-4379-9528-ff3df2152fd8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:50.267995 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:50.267995 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:50.267995 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:50.267995 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:50.267995 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:50.267995 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:50.267995 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:50.267995 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:50.267995 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:50.267995 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:50.267995 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:50.267995 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:50.267995 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:50.267995 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:50.267995 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:50.268920 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:50.268920 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:50.268920 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:50.268920 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:50.268920 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:50.268920 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:50.268920 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:50.268920 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:50.268920 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:50.268920 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:50.268920 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:50.268920 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:50.268920 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:50.268920 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:50.268920 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:50.268920 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:50.270399 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:50.270399 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:50.270399 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:50.270399 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:50.270399 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:50.270399 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:50.270399 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:50.270399 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:50.270399 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:50.270399 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:50.270399 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:50.270399 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:50.273419 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 205/400] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:50.274806 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc7776-8cb1-479d-8268-a233a0081f7c None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:50.275254 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc7776-8cb1-479d-8268-a233a0081f7c None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:50.275672 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc7776-8cb1-479d-8268-a233a0081f7c None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:50.277313 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 206/401] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:50.282894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc7776-8cb1-479d-8268-a233a0081f7c None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:50.294487 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:50.314394 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e925871-3bc8-4419-a239-84a107c3e407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:50.315200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e925871-3bc8-4419-a239-84a107c3e407 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:50.315568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e925871-3bc8-4419-a239-84a107c3e407 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:50.315931 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e925871-3bc8-4419-a239-84a107c3e407 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:50.316836 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e925871-3bc8-4419-a239-84a107c3e407 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:50.317237 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e925871-3bc8-4419-a239-84a107c3e407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:50.318821 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e925871-3bc8-4419-a239-84a107c3e407 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:50.328817 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 196/402] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:53:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:50.335254 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1abd7-37c8-4c6b-a998-5039ded47660 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:50.335686 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1abd7-37c8-4c6b-a998-5039ded47660 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:50.336056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1abd7-37c8-4c6b-a998-5039ded47660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:50.354824 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-3da1abd7-37c8-4c6b-a998-5039ded47660 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:50.356443 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-3da1abd7-37c8-4c6b-a998-5039ded47660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:50.408955 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc7776-8cb1-479d-8268-a233a0081f7c None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:50.427245 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 197/403] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:50.428717 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1abd7-37c8-4c6b-a998-5039ded47660 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:50.446808 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439f523-ff3c-4476-8b76-e6e8755e3aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:50.447877 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439f523-ff3c-4476-8b76-e6e8755e3aab None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:50.448735 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439f523-ff3c-4476-8b76-e6e8755e3aab None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:50.449155 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439f523-ff3c-4476-8b76-e6e8755e3aab None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:50.450073 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439f523-ff3c-4476-8b76-e6e8755e3aab None admin] RBAC: Adding query filter params (name=member) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:50.450536 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439f523-ff3c-4476-8b76-e6e8755e3aab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:50.452594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439f523-ff3c-4476-8b76-e6e8755e3aab None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:50.465255 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 207/404] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:50.470189 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439f523-ff3c-4476-8b76-e6e8755e3aab None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:50.515862 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:50.554042 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1abd7-37c8-4c6b-a998-5039ded47660 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:50.567150 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:50.602977 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab143c51-4615-4a89-b130-f5ea9d0fa6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:50.603679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab143c51-4615-4a89-b130-f5ea9d0fa6cd None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:50.603962 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab143c51-4615-4a89-b130-f5ea9d0fa6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:50.604236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab143c51-4615-4a89-b130-f5ea9d0fa6cd None admin] PATH_INFO: `/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:50.608159 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab143c51-4615-4a89-b130-f5ea9d0fa6cd None admin] RBAC: Authorizing `identity:get_group(group_id=2e14b4b0fee94205981b90d7ddedbc9e)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:50.609611 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab143c51-4615-4a89-b130-f5ea9d0fa6cd None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:50.613508 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 198/405] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 14 01:53:50 2022] GET /identity/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:50.619209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab143c51-4615-4a89-b130-f5ea9d0fa6cd None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:50.667207 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439f523-ff3c-4476-8b76-e6e8755e3aab None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:50.713330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d400faf-2321-46d1-9b4a-4d47507cc8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:50.714346 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d400faf-2321-46d1-9b4a-4d47507cc8ef None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:50.716331 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d400faf-2321-46d1-9b4a-4d47507cc8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:50.716846 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d400faf-2321-46d1-9b4a-4d47507cc8ef None admin] PATH_INFO: `/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:50.719380 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d400faf-2321-46d1-9b4a-4d47507cc8ef None admin] RBAC: Authorizing `identity:get_group(group_id=2e14b4b0fee94205981b90d7ddedbc9e)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:50.723058 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d400faf-2321-46d1-9b4a-4d47507cc8ef None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:50.727114 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 208/406] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 14 01:53:50 2022] GET /identity/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e => generated 226 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:50.732851 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d400faf-2321-46d1-9b4a-4d47507cc8ef None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:50.808990 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab143c51-4615-4a89-b130-f5ea9d0fa6cd None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:50.845825 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad148f89-09bc-4901-aa23-c8deb371cc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:50.846485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad148f89-09bc-4901-aa23-c8deb371cc98 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:50.846766 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad148f89-09bc-4901-aa23-c8deb371cc98 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:50.847034 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad148f89-09bc-4901-aa23-c8deb371cc98 None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:50.848188 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad148f89-09bc-4901-aa23-c8deb371cc98 None admin] RBAC: Authorizing `identity:get_project(project_id=60ea1e9a961b4e3b99b9fe44ac610808)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:50.849637 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad148f89-09bc-4901-aa23-c8deb371cc98 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:50.851824 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 199/407] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:50 2022] GET /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:50.852668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad148f89-09bc-4901-aa23-c8deb371cc98 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:50.867744 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:50.873942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d400faf-2321-46d1-9b4a-4d47507cc8ef None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:50.906604 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:50.917740 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f435971-f739-4120-b14c-ba4c7a61ea0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:50.919057 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f435971-f739-4120-b14c-ba4c7a61ea0f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:50.919752 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f435971-f739-4120-b14c-ba4c7a61ea0f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:50.920225 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f435971-f739-4120-b14c-ba4c7a61ea0f None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:50.922194 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f435971-f739-4120-b14c-ba4c7a61ea0f None admin] RBAC: Authorizing `identity:get_project(project_id=60ea1e9a961b4e3b99b9fe44ac610808)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:50.924790 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f435971-f739-4120-b14c-ba4c7a61ea0f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:50.932618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 209/408] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:50 2022] GET /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:50.935264 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f435971-f739-4120-b14c-ba4c7a61ea0f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:50.975694 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad148f89-09bc-4901-aa23-c8deb371cc98 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:51.012532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c722a8a4-2377-4780-b466-e0f4b273e051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:51.013161 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722a8a4-2377-4780-b466-e0f4b273e051 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:51.013434 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722a8a4-2377-4780-b466-e0f4b273e051 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:51.013700 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722a8a4-2377-4780-b466-e0f4b273e051 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:51.025864 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c722a8a4-2377-4780-b466-e0f4b273e051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:51.033123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-c722a8a4-2377-4780-b466-e0f4b273e051 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:53:51.033123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:51.033123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:51.033123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:51.033123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:51.033123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:51.033123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:51.033123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:51.033123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:51.033123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:51.033123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:51.033123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:51.033123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:51.033123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:51.033123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:51.033123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:51.034066 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:51.034066 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:51.034066 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:51.034066 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:51.034066 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:51.034066 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:51.034066 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:51.034066 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:51.034066 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:51.034066 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:51.034066 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:51.034066 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:51.034066 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:51.034066 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:51.034066 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:51.034066 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:51.034924 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:51.034924 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:51.034924 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:51.034924 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:51.034924 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:51.034924 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:51.034924 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:51.034924 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:51.034924 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:51.034924 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:51.034924 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:53:51.034924 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:51.042378 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 200/409] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:53:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:51.042378 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c722a8a4-2377-4780-b466-e0f4b273e051 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:51.066776 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f435971-f739-4120-b14c-ba4c7a61ea0f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:51.114295 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbb7e2-6d4b-4dae-8377-fe79605c3ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:51.116101 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbb7e2-6d4b-4dae-8377-fe79605c3ff5 None admin] REQUEST_METHOD: `PUT` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:51.116738 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbb7e2-6d4b-4dae-8377-fe79605c3ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:51.117359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbb7e2-6d4b-4dae-8377-fe79605c3ff5 None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808/groups/2e14b4b0fee94205981b90d7ddedbc9e/roles/e842ac50986946a4971a51a06a39e16b` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:51.120792 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fbb7e2-6d4b-4dae-8377-fe79605c3ff5 None admin] RBAC: Authorizing `identity:create_grant(project_id=60ea1e9a961b4e3b99b9fe44ac610808, group_id=2e14b4b0fee94205981b90d7ddedbc9e, role_id=e842ac50986946a4971a51a06a39e16b)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:51.123420 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fbb7e2-6d4b-4dae-8377-fe79605c3ff5 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:51.140438 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 210/410] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Sep 14 01:53:50 2022] PUT /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808/groups/2e14b4b0fee94205981b90d7ddedbc9e/roles/e842ac50986946a4971a51a06a39e16b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:53:51.142114 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbb7e2-6d4b-4dae-8377-fe79605c3ff5 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:51.178140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c722a8a4-2377-4780-b466-e0f4b273e051 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:51.182655 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:51.192882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:51.234551 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106af604-1bea-4909-917c-29619f70fb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:51.235953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106af604-1bea-4909-917c-29619f70fb71 None admin] REQUEST_METHOD: `PUT` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:51.237289 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106af604-1bea-4909-917c-29619f70fb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:51.237772 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106af604-1bea-4909-917c-29619f70fb71 None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808/groups/2e14b4b0fee94205981b90d7ddedbc9e/roles/934ee0e8483c496cb8813eccbe0ebf55` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:51.240116 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106af604-1bea-4909-917c-29619f70fb71 None admin] RBAC: Authorizing `identity:create_grant(project_id=60ea1e9a961b4e3b99b9fe44ac610808, group_id=2e14b4b0fee94205981b90d7ddedbc9e, role_id=934ee0e8483c496cb8813eccbe0ebf55)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:51.242327 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106af604-1bea-4909-917c-29619f70fb71 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:51.264198 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 201/411] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Sep 14 01:53:51 2022] PUT /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808/groups/2e14b4b0fee94205981b90d7ddedbc9e/roles/934ee0e8483c496cb8813eccbe0ebf55 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:53:51.276704 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbb7e2-6d4b-4dae-8377-fe79605c3ff5 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:51.317676 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47120d79-9c3e-48f3-a86a-c7133fcb57fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:51.318359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47120d79-9c3e-48f3-a86a-c7133fcb57fe None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:51.318586 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47120d79-9c3e-48f3-a86a-c7133fcb57fe None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:51.318826 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47120d79-9c3e-48f3-a86a-c7133fcb57fe None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:51.319516 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47120d79-9c3e-48f3-a86a-c7133fcb57fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:51.319816 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47120d79-9c3e-48f3-a86a-c7133fcb57fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:51.321219 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47120d79-9c3e-48f3-a86a-c7133fcb57fe None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:51.330971 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 211/412] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:51.337050 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106af604-1bea-4909-917c-29619f70fb71 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:51.466017 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106af604-1bea-4909-917c-29619f70fb71 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:51.503068 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c39d52-61eb-4e38-bd43-8bd521c9a180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:51.504608 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c39d52-61eb-4e38-bd43-8bd521c9a180 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:51.505092 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c39d52-61eb-4e38-bd43-8bd521c9a180 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:51.505567 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c39d52-61eb-4e38-bd43-8bd521c9a180 None admin] PATH_INFO: `/v3/groups/c4e92f5e40544357b755938ebc5b8a05` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:51.507190 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c39d52-61eb-4e38-bd43-8bd521c9a180 None admin] RBAC: Authorizing `identity:get_group(group_id=c4e92f5e40544357b755938ebc5b8a05)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:51.508881 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c39d52-61eb-4e38-bd43-8bd521c9a180 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:51.510703 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 202/413] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 14 01:53:51 2022] GET /identity/v3/groups/c4e92f5e40544357b755938ebc5b8a05 => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:51.516568 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47120d79-9c3e-48f3-a86a-c7133fcb57fe None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:51.632775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:51.646990 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47120d79-9c3e-48f3-a86a-c7133fcb57fe None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:51.696584 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d0eb6-2bef-4393-a616-7a745354ebd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:51.697596 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d0eb6-2bef-4393-a616-7a745354ebd3 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:51.698106 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d0eb6-2bef-4393-a616-7a745354ebd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:51.698580 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d0eb6-2bef-4393-a616-7a745354ebd3 None admin] PATH_INFO: `/v3/projects/2ca72c26a2124532970b02d36e64c3f9` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:51.699954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d0eb6-2bef-4393-a616-7a745354ebd3 None admin] RBAC: Authorizing `identity:get_project(project_id=2ca72c26a2124532970b02d36e64c3f9)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:51.701576 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0023e-3d31-4a68-8c63-7e859016eeff None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:51.702089 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0023e-3d31-4a68-8c63-7e859016eeff None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:51.702573 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0023e-3d31-4a68-8c63-7e859016eeff None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:51.704149 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 203/414] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:51.705868 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d0eb6-2bef-4393-a616-7a745354ebd3 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:51.708603 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 212/415] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:51 2022] GET /identity/v3/projects/2ca72c26a2124532970b02d36e64c3f9 => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:51.709975 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2965bd-1617-44ff-a30b-7153cefee718 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:51.710483 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2965bd-1617-44ff-a30b-7153cefee718 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:51.710954 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2965bd-1617-44ff-a30b-7153cefee718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:51.719246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d0eb6-2bef-4393-a616-7a745354ebd3 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:51.742268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-8e2965bd-1617-44ff-a30b-7153cefee718 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:51.748491 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-8e2965bd-1617-44ff-a30b-7153cefee718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:51.795054 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:51.854261 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d0eb6-2bef-4393-a616-7a745354ebd3 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:51.866618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 204/416] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:53:51.891140 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733d03c-ba8e-41d4-a34b-75c7a61d0051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:51.892127 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733d03c-ba8e-41d4-a34b-75c7a61d0051 None admin] REQUEST_METHOD: `PUT` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:51.892610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733d03c-ba8e-41d4-a34b-75c7a61d0051 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:51.893101 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733d03c-ba8e-41d4-a34b-75c7a61d0051 None admin] PATH_INFO: `/v3/projects/2ca72c26a2124532970b02d36e64c3f9/groups/c4e92f5e40544357b755938ebc5b8a05/roles/aa420c2fb1ab4bb3b05b004a4d89779a` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:51.895914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733d03c-ba8e-41d4-a34b-75c7a61d0051 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ca72c26a2124532970b02d36e64c3f9, group_id=c4e92f5e40544357b755938ebc5b8a05, role_id=aa420c2fb1ab4bb3b05b004a4d89779a)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:51.897575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733d03c-ba8e-41d4-a34b-75c7a61d0051 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:51.920974 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 213/417] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Sep 14 01:53:51 2022] PUT /identity/v3/projects/2ca72c26a2124532970b02d36e64c3f9/groups/c4e92f5e40544357b755938ebc5b8a05/roles/aa420c2fb1ab4bb3b05b004a4d89779a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:53:52.242186 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2965bd-1617-44ff-a30b-7153cefee718 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:52.403941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2965bd-1617-44ff-a30b-7153cefee718 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:52.460492 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a048951e-75ae-417c-8077-c57019b61f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:52.461616 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048951e-75ae-417c-8077-c57019b61f75 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:52.462200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048951e-75ae-417c-8077-c57019b61f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:52.462682 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048951e-75ae-417c-8077-c57019b61f75 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:52.482988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a048951e-75ae-417c-8077-c57019b61f75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:52.488863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-a048951e-75ae-417c-8077-c57019b61f75 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:52.488863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:52.488863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:52.488863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:52.488863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:52.488863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:52.488863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:52.488863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:52.488863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:52.488863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:52.488863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:52.488863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:52.488863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:52.488863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:52.488863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:52.488863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:52.489756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:52.489756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:52.489756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:52.489756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:52.489756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:52.489756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:52.489756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:52.489756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:52.489756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:52.489756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:52.489756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:52.489756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:52.489756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:52.489756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:52.489756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:52.489756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:52.490618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:52.490618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:52.490618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:52.490618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:52.490618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:52.490618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:52.490618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:52.490618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:52.490618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:52.490618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:52.490618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:52.490618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:52.493315 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 205/418] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:53:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:52.500353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733d03c-ba8e-41d4-a34b-75c7a61d0051 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:52.524652 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:52.575146 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543538ee-c250-4f6d-bc87-6b50461f26cc None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:52.575506 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543538ee-c250-4f6d-bc87-6b50461f26cc None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:52.575690 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543538ee-c250-4f6d-bc87-6b50461f26cc None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:52.577768 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 206/419] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:53:52.584035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838edc1-632e-40da-9332-2df7b5420021 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:52.584319 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838edc1-632e-40da-9332-2df7b5420021 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:52.584594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838edc1-632e-40da-9332-2df7b5420021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:52.610519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-9838edc1-632e-40da-9332-2df7b5420021 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:52.611720 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-9838edc1-632e-40da-9332-2df7b5420021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:52.647687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:52.660588 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733d03c-ba8e-41d4-a34b-75c7a61d0051 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:52.727115 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5346a-2640-4747-988e-c55aee248846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:52.728131 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5346a-2640-4747-988e-c55aee248846 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:52.728611 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5346a-2640-4747-988e-c55aee248846 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:52.729115 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5346a-2640-4747-988e-c55aee248846 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:52.729971 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a5346a-2640-4747-988e-c55aee248846 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:52.730657 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a5346a-2640-4747-988e-c55aee248846 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:52.733280 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a5346a-2640-4747-988e-c55aee248846 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:52.745298 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 214/420] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:53:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:52.751356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 207/421] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:53:52.751629 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5346a-2640-4747-988e-c55aee248846 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:52.831692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:52.854276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5346a-2640-4747-988e-c55aee248846 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:52.890438 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d58d6-4f1b-48e0-991b-6b9d7fc18e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:52.891410 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d58d6-4f1b-48e0-991b-6b9d7fc18e8a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:52.891908 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d58d6-4f1b-48e0-991b-6b9d7fc18e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:52.892373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d58d6-4f1b-48e0-991b-6b9d7fc18e8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:52.965226 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d58d6-4f1b-48e0-991b-6b9d7fc18e8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:52.967054 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d58d6-4f1b-48e0-991b-6b9d7fc18e8a None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:53.046593 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:53.074338 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 215/422] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:53.077470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838edc1-632e-40da-9332-2df7b5420021 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:53.163980 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d58d6-4f1b-48e0-991b-6b9d7fc18e8a None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:53.196791 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838edc1-632e-40da-9332-2df7b5420021 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:53.235597 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05475a0-0fb5-4a1f-acd3-efcfb5f04bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:53.236238 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05475a0-0fb5-4a1f-acd3-efcfb5f04bce None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:53.236517 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05475a0-0fb5-4a1f-acd3-efcfb5f04bce None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:53.236789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05475a0-0fb5-4a1f-acd3-efcfb5f04bce None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:53.238716 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05475a0-0fb5-4a1f-acd3-efcfb5f04bce None admin] RBAC: Authorizing `identity:get_project(project_id=f8542c9d54464124a730d0ce2363e406)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:53.240167 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05475a0-0fb5-4a1f-acd3-efcfb5f04bce None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:53.241797 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 208/423] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:53 2022] GET /identity/v3/projects/f8542c9d54464124a730d0ce2363e406 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:53.248975 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05475a0-0fb5-4a1f-acd3-efcfb5f04bce None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:53.312628 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d58d6-4f1b-48e0-991b-6b9d7fc18e8a None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:53.323749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:53.353869 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c057b-1d9b-46e2-8d60-e399fc73cf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:53.356706 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c057b-1d9b-46e2-8d60-e399fc73cf34 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:53.357190 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c057b-1d9b-46e2-8d60-e399fc73cf34 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:53.357656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c057b-1d9b-46e2-8d60-e399fc73cf34 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:53.374268 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c057b-1d9b-46e2-8d60-e399fc73cf34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:53.380186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-8c0c057b-1d9b-46e2-8d60-e399fc73cf34 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:53.380186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:53.380186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:53.380186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:53.380186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:53.380186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:53.380186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:53.380186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:53.380186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:53.380186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:53.380186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:53.380186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:53.380186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:53.380186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:53.380186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:53.380186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:53.381102 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:53.381102 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:53.381102 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:53.381102 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:53.381102 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:53.381102 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:53.381102 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:53.381102 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:53.381102 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:53.381102 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:53.381102 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:53.381102 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:53.381102 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:53.381102 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:53.381102 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:53.381102 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:53.382244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:53.382244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:53.382244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:53.382244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:53.382244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:53.382244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:53.382244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:53.382244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:53.382244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:53.382244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:53.382244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:53.382244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:53.384913 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 216/424] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:53.392112 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c057b-1d9b-46e2-8d60-e399fc73cf34 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:53.442218 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05475a0-0fb5-4a1f-acd3-efcfb5f04bce None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:53.476946 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ce4c2-f202-4675-af51-95e3e3639a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:53.477728 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ce4c2-f202-4675-af51-95e3e3639a49 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:53.478025 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ce4c2-f202-4675-af51-95e3e3639a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:53.478298 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ce4c2-f202-4675-af51-95e3e3639a49 None admin] PATH_INFO: `/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:53.479418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ce4c2-f202-4675-af51-95e3e3639a49 None admin] RBAC: Authorizing `identity:get_group(group_id=2e14b4b0fee94205981b90d7ddedbc9e)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:53.480956 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ce4c2-f202-4675-af51-95e3e3639a49 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:53.482958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 209/425] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 14 01:53:53 2022] GET /identity/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:53.493201 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:53.494181 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ce4c2-f202-4675-af51-95e3e3639a49 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:53.603120 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c057b-1d9b-46e2-8d60-e399fc73cf34 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:53.658402 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:53.668925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ce4c2-f202-4675-af51-95e3e3639a49 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:53.691326 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea349f3-69a2-41c8-b4f0-bdd84498c155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:53.692177 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea349f3-69a2-41c8-b4f0-bdd84498c155 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:53.692634 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea349f3-69a2-41c8-b4f0-bdd84498c155 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:53.693065 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea349f3-69a2-41c8-b4f0-bdd84498c155 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:53.693851 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea349f3-69a2-41c8-b4f0-bdd84498c155 None admin] RBAC: Adding query filter params (name=member) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:53.694196 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea349f3-69a2-41c8-b4f0-bdd84498c155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:53.695619 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea349f3-69a2-41c8-b4f0-bdd84498c155 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:53.705181 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334b151-0ed5-42e5-ba9c-ec2bdf9a3f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:53.706118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334b151-0ed5-42e5-ba9c-ec2bdf9a3f7f None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:53.706604 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334b151-0ed5-42e5-ba9c-ec2bdf9a3f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:53.707532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334b151-0ed5-42e5-ba9c-ec2bdf9a3f7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:53.708444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334b151-0ed5-42e5-ba9c-ec2bdf9a3f7f None admin] RBAC: Adding query filter params (group.id=2e14b4b0fee94205981b90d7ddedbc9e, role.id=e842ac50986946a4971a51a06a39e16b, scope.project.id=f8542c9d54464124a730d0ce2363e406) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:53.708968 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334b151-0ed5-42e5-ba9c-ec2bdf9a3f7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:53.711230 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334b151-0ed5-42e5-ba9c-ec2bdf9a3f7f None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:53.717685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 217/426] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:53.718417 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 210/427] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Sep 14 01:53:53 2022] GET /identity/v3/role_assignments?group.id=2e14b4b0fee94205981b90d7ddedbc9e&scope.project.id=f8542c9d54464124a730d0ce2363e406&role.id=e842ac50986946a4971a51a06a39e16b => generated 611 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:53.723528 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea349f3-69a2-41c8-b4f0-bdd84498c155 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:53.884716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:53.888591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea349f3-69a2-41c8-b4f0-bdd84498c155 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:53.929600 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ba54b-24fa-40f0-a329-fec5b14cac56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:53.930268 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ba54b-24fa-40f0-a329-fec5b14cac56 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:53.930551 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ba54b-24fa-40f0-a329-fec5b14cac56 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:53.930824 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ba54b-24fa-40f0-a329-fec5b14cac56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:54.010209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aac14c-fed3-4765-8945-2cfe5b4f7639 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:54.010829 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aac14c-fed3-4765-8945-2cfe5b4f7639 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:54.011298 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aac14c-fed3-4765-8945-2cfe5b4f7639 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:54.013059 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 211/428] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:54.017751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ba54b-24fa-40f0-a329-fec5b14cac56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:54.018974 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f92dee-348a-4cd5-b017-c0ad5a62e83b None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:54.019814 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ba54b-24fa-40f0-a329-fec5b14cac56 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:54.020250 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f92dee-348a-4cd5-b017-c0ad5a62e83b None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:54.020715 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f92dee-348a-4cd5-b017-c0ad5a62e83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:54.043540 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-a8f92dee-348a-4cd5-b017-c0ad5a62e83b None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:54.045069 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-a8f92dee-348a-4cd5-b017-c0ad5a62e83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:54.132401 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:54.135912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 212/429] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:54.137231 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 218/430] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:54.143122 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f92dee-348a-4cd5-b017-c0ad5a62e83b None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:54.225965 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c11908-501a-4a20-97c6-870def3ed5ba None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:54.226353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c11908-501a-4a20-97c6-870def3ed5ba None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:54.226544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c11908-501a-4a20-97c6-870def3ed5ba None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:54.228128 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 219/431] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:53:54.233082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ad70a-06d0-40f7-bbf1-984383a656d6 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:54.233364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ad70a-06d0-40f7-bbf1-984383a656d6 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:54.233638 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ad70a-06d0-40f7-bbf1-984383a656d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:54.240937 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:54.253369 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-581ad70a-06d0-40f7-bbf1-984383a656d6 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:54.255381 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-581ad70a-06d0-40f7-bbf1-984383a656d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:54.262831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f92dee-348a-4cd5-b017-c0ad5a62e83b None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:54.300256 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6414bf-984d-4aeb-b324-8d893cb87ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:54.301280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6414bf-984d-4aeb-b324-8d893cb87ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:54.301757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6414bf-984d-4aeb-b324-8d893cb87ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:54.302277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6414bf-984d-4aeb-b324-8d893cb87ff9 None admin] PATH_INFO: `/v3/projects/f8542c9d54464124a730d0ce2363e406` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:54.303732 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6414bf-984d-4aeb-b324-8d893cb87ff9 None admin] RBAC: Authorizing `identity:get_project(project_id=f8542c9d54464124a730d0ce2363e406)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:54.305416 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6414bf-984d-4aeb-b324-8d893cb87ff9 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:54.307730 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 213/432] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:54 2022] GET /identity/v3/projects/f8542c9d54464124a730d0ce2363e406 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:54.318012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6414bf-984d-4aeb-b324-8d893cb87ff9 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:54.345881 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 220/433] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:53:54.418989 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6414bf-984d-4aeb-b324-8d893cb87ff9 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:54.434874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ad70a-06d0-40f7-bbf1-984383a656d6 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:54.456839 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c1aca-23a6-48b6-96c4-5c7e05d4347f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:54.457473 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c1aca-23a6-48b6-96c4-5c7e05d4347f None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:54.457753 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c1aca-23a6-48b6-96c4-5c7e05d4347f None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:54.458045 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c1aca-23a6-48b6-96c4-5c7e05d4347f None admin] PATH_INFO: `/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:54.459568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c1aca-23a6-48b6-96c4-5c7e05d4347f None admin] RBAC: Authorizing `identity:get_group(group_id=2e14b4b0fee94205981b90d7ddedbc9e)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:54.460997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c1aca-23a6-48b6-96c4-5c7e05d4347f None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:54.464224 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 214/434] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 14 01:53:54 2022] GET /identity/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:54.468711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c1aca-23a6-48b6-96c4-5c7e05d4347f None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:54.492830 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:54.562547 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ad70a-06d0-40f7-bbf1-984383a656d6 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:54.578209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c1aca-23a6-48b6-96c4-5c7e05d4347f None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:54.596142 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:54.604404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d90d1-0d65-4f73-9180-0e8339619f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:54.605891 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d90d1-0d65-4f73-9180-0e8339619f4f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:54.606418 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d90d1-0d65-4f73-9180-0e8339619f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:54.606902 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d90d1-0d65-4f73-9180-0e8339619f4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:54.614101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee26338-5b25-4443-9df8-77bd9081910f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:54.614705 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee26338-5b25-4443-9df8-77bd9081910f None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:54.614978 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee26338-5b25-4443-9df8-77bd9081910f None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:54.615242 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee26338-5b25-4443-9df8-77bd9081910f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:54.615959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee26338-5b25-4443-9df8-77bd9081910f None admin] RBAC: Adding query filter params (group.id=2e14b4b0fee94205981b90d7ddedbc9e, role.id=934ee0e8483c496cb8813eccbe0ebf55, scope.project.id=f8542c9d54464124a730d0ce2363e406) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:54.616343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee26338-5b25-4443-9df8-77bd9081910f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:54.617825 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee26338-5b25-4443-9df8-77bd9081910f None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:54.620308 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d90d1-0d65-4f73-9180-0e8339619f4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:54.623861 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 215/435] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Sep 14 01:53:54 2022] GET /identity/v3/role_assignments?group.id=2e14b4b0fee94205981b90d7ddedbc9e&scope.project.id=f8542c9d54464124a730d0ce2363e406&role.id=934ee0e8483c496cb8813eccbe0ebf55 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:54.624647 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee26338-5b25-4443-9df8-77bd9081910f None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:54.626111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-eb8d90d1-0d65-4f73-9180-0e8339619f4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:54.626111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:54.626111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:54.626111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:54.626111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:54.626111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:54.626111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:54.626111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:54.626111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:54.626111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:54.626111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:54.626111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:54.626111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:54.626111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:54.626111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:54.626111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:54.627108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:54.627108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:54.627108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:54.627108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:54.627108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:54.627108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:54.627108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:54.627108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:54.627108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:54.627108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:54.627108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:54.627108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:54.627108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:54.627108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:54.627108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:54.627108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:54.627930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:54.627930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:54.627930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:54.627930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:54.627930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:54.627930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:54.627930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:54.627930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:54.627930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:54.627930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:54.627930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:53:54.627930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:54.630021 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 221/436] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:53:54.631729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ae007-1606-42d6-bdeb-839f64c891e7 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:54.632014 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ae007-1606-42d6-bdeb-839f64c891e7 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:54.632288 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ae007-1606-42d6-bdeb-839f64c891e7 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:54.633765 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 222/437] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:54.636031 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ae007-1606-42d6-bdeb-839f64c891e7 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:54.722407 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:54.740376 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee26338-5b25-4443-9df8-77bd9081910f None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:54.755306 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ae007-1606-42d6-bdeb-839f64c891e7 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:54.780785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db757aa1-a23e-420a-9472-e6b1a75786bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:54.781440 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db757aa1-a23e-420a-9472-e6b1a75786bf None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:54.781719 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db757aa1-a23e-420a-9472-e6b1a75786bf None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:54.782007 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db757aa1-a23e-420a-9472-e6b1a75786bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:54.794376 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5253bc-7a9b-4fc4-bdfe-ed525eb18238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:54.794913 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db757aa1-a23e-420a-9472-e6b1a75786bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:54.795037 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5253bc-7a9b-4fc4-bdfe-ed525eb18238 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:54.795287 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5253bc-7a9b-4fc4-bdfe-ed525eb18238 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:54.795517 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5253bc-7a9b-4fc4-bdfe-ed525eb18238 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:54.796172 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5253bc-7a9b-4fc4-bdfe-ed525eb18238 None admin] RBAC: Adding query filter params (name=member) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:54.796470 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5253bc-7a9b-4fc4-bdfe-ed525eb18238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:54.797842 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5253bc-7a9b-4fc4-bdfe-ed525eb18238 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:54.800321 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-db757aa1-a23e-420a-9472-e6b1a75786bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:54.800321 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:54.800321 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:54.800321 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:54.800321 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:54.800321 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:54.800321 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:54.800321 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:54.800321 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:54.800321 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:54.800321 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:54.800321 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:54.800321 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:54.800321 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:54.800321 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:54.800321 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:54.801753 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:54.801753 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:54.801753 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:54.801753 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:54.801753 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:54.801753 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:54.801753 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:54.801753 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:54.801753 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:54.801753 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:54.801753 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:54.801753 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:54.801753 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:54.801753 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:54.801753 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:54.801753 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:54.802615 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:54.802615 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:54.802615 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:54.802615 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:54.802615 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:54.802615 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:54.802615 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:54.802615 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:54.802615 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:54.802615 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:54.802615 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:53:54.802615 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:54.802615 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 216/438] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:53:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:54.803282 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0fac4-2c19-4806-b3f2-d1535d5e8e54 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:54.803827 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0fac4-2c19-4806-b3f2-d1535d5e8e54 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:54.803827 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0fac4-2c19-4806-b3f2-d1535d5e8e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:54.809560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 223/439] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:53:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:54.810679 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5253bc-7a9b-4fc4-bdfe-ed525eb18238 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:54.823633 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-b8e0fac4-2c19-4806-b3f2-d1535d5e8e54 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:54.824678 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-b8e0fac4-2c19-4806-b3f2-d1535d5e8e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:54.870609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:54.904972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 217/440] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:54.905157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0fac4-2c19-4806-b3f2-d1535d5e8e54 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:54.927953 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5253bc-7a9b-4fc4-bdfe-ed525eb18238 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:54.980705 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58584f86-774d-473e-8ada-3bc1d20131df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:54.981349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58584f86-774d-473e-8ada-3bc1d20131df None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:54.981694 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58584f86-774d-473e-8ada-3bc1d20131df None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:54.982086 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58584f86-774d-473e-8ada-3bc1d20131df None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:54.983322 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58584f86-774d-473e-8ada-3bc1d20131df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:54.983322 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58584f86-774d-473e-8ada-3bc1d20131df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:54.984535 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58584f86-774d-473e-8ada-3bc1d20131df None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:54.997198 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 224/441] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:53:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:55.004239 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58584f86-774d-473e-8ada-3bc1d20131df None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:55.013317 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:55.043804 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0fac4-2c19-4806-b3f2-d1535d5e8e54 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:55.083594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e761c-0080-4bef-8716-96743d9462b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:55.084207 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e761c-0080-4bef-8716-96743d9462b6 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:55.084481 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e761c-0080-4bef-8716-96743d9462b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:55.084744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e761c-0080-4bef-8716-96743d9462b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:55.139082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58584f86-774d-473e-8ada-3bc1d20131df None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:55.155256 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:55.178045 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e761c-0080-4bef-8716-96743d9462b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:55.179895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e761c-0080-4bef-8716-96743d9462b6 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:55.180884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46640b5-ecf3-47aa-b7bf-54af0904c46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:55.181506 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46640b5-ecf3-47aa-b7bf-54af0904c46a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:55.181958 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46640b5-ecf3-47aa-b7bf-54af0904c46a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:55.182329 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46640b5-ecf3-47aa-b7bf-54af0904c46a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:55.261346 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46640b5-ecf3-47aa-b7bf-54af0904c46a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:55.262767 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46640b5-ecf3-47aa-b7bf-54af0904c46a None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:55.274676 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:55.309650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 218/442] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:55.310846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e761c-0080-4bef-8716-96743d9462b6 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:55.367417 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 225/443] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:55.368292 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46640b5-ecf3-47aa-b7bf-54af0904c46a None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:55.383700 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:55.417211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e761c-0080-4bef-8716-96743d9462b6 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:55.449672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2828b9-a4d0-482b-b6f8-35791881b924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:55.450315 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2828b9-a4d0-482b-b6f8-35791881b924 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:55.450589 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2828b9-a4d0-482b-b6f8-35791881b924 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:55.450855 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2828b9-a4d0-482b-b6f8-35791881b924 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:55.461206 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2828b9-a4d0-482b-b6f8-35791881b924 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:53:55.466035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-6c2828b9-a4d0-482b-b6f8-35791881b924 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:53:55.466035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:55.466035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:55.466035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:55.466035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:55.466035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:55.466035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:55.466035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:55.466035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:55.466035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:55.466035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:55.466035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:55.466035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:53:55.466035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:53:55.466035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:53:55.466035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:53:55.467063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:53:55.467063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:53:55.467063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:53:55.467063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:53:55.467063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:55.467063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:55.467063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:55.467063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:55.467063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:55.467063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:55.467063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:55.467063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:55.467063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:55.467063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:55.467063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:55.467063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:55.467892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:55.467892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:55.467892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:55.467892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:55.467892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:53:55.467892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:53:55.467892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:53:55.467892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:53:55.467892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:53:55.467892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:53:55.467892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:53:55.467892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:55.467892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 219/444] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:53:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:55.468579 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2828b9-a4d0-482b-b6f8-35791881b924 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:55.474329 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46640b5-ecf3-47aa-b7bf-54af0904c46a None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:55.501045 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:55.508296 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b331f-c07f-4948-af62-7741edc2c713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:55.508926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b331f-c07f-4948-af62-7741edc2c713 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:55.509206 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b331f-c07f-4948-af62-7741edc2c713 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:55.509479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b331f-c07f-4948-af62-7741edc2c713 None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:55.510597 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b331f-c07f-4948-af62-7741edc2c713 None admin] RBAC: Authorizing `identity:get_project(project_id=60ea1e9a961b4e3b99b9fe44ac610808)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:55.512082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b331f-c07f-4948-af62-7741edc2c713 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:55.513809 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 226/445] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:55 2022] GET /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:55.514423 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b331f-c07f-4948-af62-7741edc2c713 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:55.569722 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2828b9-a4d0-482b-b6f8-35791881b924 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:55.593462 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:55.603129 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e73fb-67b8-4d73-bca6-55d15379575c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:55.603743 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e73fb-67b8-4d73-bca6-55d15379575c None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:55.604014 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e73fb-67b8-4d73-bca6-55d15379575c None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:55.604278 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e73fb-67b8-4d73-bca6-55d15379575c None admin] PATH_INFO: `/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:55.605364 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e73fb-67b8-4d73-bca6-55d15379575c None admin] RBAC: Authorizing `identity:get_project(project_id=60ea1e9a961b4e3b99b9fe44ac610808)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:55.606809 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e73fb-67b8-4d73-bca6-55d15379575c None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:55.608394 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 220/446] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:55 2022] GET /identity/v3/projects/60ea1e9a961b4e3b99b9fe44ac610808 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:55.609143 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e73fb-67b8-4d73-bca6-55d15379575c None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:55.611645 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b331f-c07f-4948-af62-7741edc2c713 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:55.653302 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532124fd-5a4e-411e-b835-16aeda779b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:55.653911 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532124fd-5a4e-411e-b835-16aeda779b1c None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:55.654218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532124fd-5a4e-411e-b835-16aeda779b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:55.654486 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532124fd-5a4e-411e-b835-16aeda779b1c None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:55.655134 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532124fd-5a4e-411e-b835-16aeda779b1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:55.655432 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532124fd-5a4e-411e-b835-16aeda779b1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:55.656791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532124fd-5a4e-411e-b835-16aeda779b1c None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:55.664803 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 227/447] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:53:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:55.665415 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532124fd-5a4e-411e-b835-16aeda779b1c None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:55.711756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e73fb-67b8-4d73-bca6-55d15379575c None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:55.713229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:55.745852 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdd279-f66a-4fc6-8052-e81c6ffdd6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:55.746497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdd279-f66a-4fc6-8052-e81c6ffdd6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:55.746770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdd279-f66a-4fc6-8052-e81c6ffdd6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:55.747036 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdd279-f66a-4fc6-8052-e81c6ffdd6b0 None admin] PATH_INFO: `/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:55.748129 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfdd279-f66a-4fc6-8052-e81c6ffdd6b0 None admin] RBAC: Authorizing `identity:get_group(group_id=2e14b4b0fee94205981b90d7ddedbc9e)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:55.749548 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfdd279-f66a-4fc6-8052-e81c6ffdd6b0 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:55.751117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 221/448] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 14 01:53:55 2022] GET /identity/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:55.751864 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdd279-f66a-4fc6-8052-e81c6ffdd6b0 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:55.761683 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532124fd-5a4e-411e-b835-16aeda779b1c None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:55.794573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f2b57-cd6d-4d51-bb91-4566a9ebbd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:55.795125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f2b57-cd6d-4d51-bb91-4566a9ebbd40 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:55.795403 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f2b57-cd6d-4d51-bb91-4566a9ebbd40 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:55.795676 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f2b57-cd6d-4d51-bb91-4566a9ebbd40 None admin] PATH_INFO: `/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:55.797056 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f2b57-cd6d-4d51-bb91-4566a9ebbd40 None admin] RBAC: Authorizing `identity:get_group(group_id=2e14b4b0fee94205981b90d7ddedbc9e)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:55.798530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f2b57-cd6d-4d51-bb91-4566a9ebbd40 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:55.800246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 228/449] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 14 01:53:55 2022] GET /identity/v3/groups/2e14b4b0fee94205981b90d7ddedbc9e => generated 226 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:55.801571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f2b57-cd6d-4d51-bb91-4566a9ebbd40 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:55.820391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:55.854363 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdd279-f66a-4fc6-8052-e81c6ffdd6b0 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:55.887999 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b37265-2764-4dca-b158-f8821c50e4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:55.888585 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b37265-2764-4dca-b158-f8821c50e4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:55.888859 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b37265-2764-4dca-b158-f8821c50e4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:55.889122 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b37265-2764-4dca-b158-f8821c50e4b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:55.905129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f2b57-cd6d-4d51-bb91-4566a9ebbd40 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:55.928100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:55.939345 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0019f0d-1df6-46d5-a929-ada108933fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:55.939946 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0019f0d-1df6-46d5-a929-ada108933fac None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:55.940224 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0019f0d-1df6-46d5-a929-ada108933fac None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:55.940496 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0019f0d-1df6-46d5-a929-ada108933fac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:55.941255 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0019f0d-1df6-46d5-a929-ada108933fac None admin] RBAC: Adding query filter params (group.id=2e14b4b0fee94205981b90d7ddedbc9e, role.id=934ee0e8483c496cb8813eccbe0ebf55, scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:55.941567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0019f0d-1df6-46d5-a929-ada108933fac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:55.943032 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0019f0d-1df6-46d5-a929-ada108933fac None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:55.948698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 229/450] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Sep 14 01:53:55 2022] GET /identity/v3/role_assignments?group.id=2e14b4b0fee94205981b90d7ddedbc9e&scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808&role.id=934ee0e8483c496cb8813eccbe0ebf55 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:55.949474 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0019f0d-1df6-46d5-a929-ada108933fac None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:55.957132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b37265-2764-4dca-b158-f8821c50e4b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:55.958681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b37265-2764-4dca-b158-f8821c50e4b4 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:56.053627 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:56.070607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 222/451] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:53:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:53:56.076186 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b37265-2764-4dca-b158-f8821c50e4b4 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:56.082425 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0019f0d-1df6-46d5-a929-ada108933fac None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:56.127342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068345f-0de0-4b08-938f-251ee2053928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:56.127959 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068345f-0de0-4b08-938f-251ee2053928 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:56.128238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068345f-0de0-4b08-938f-251ee2053928 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:56.128603 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068345f-0de0-4b08-938f-251ee2053928 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:56.129329 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068345f-0de0-4b08-938f-251ee2053928 None admin] RBAC: Adding query filter params (group.id=2e14b4b0fee94205981b90d7ddedbc9e, role.id=e842ac50986946a4971a51a06a39e16b, scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:56.129635 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068345f-0de0-4b08-938f-251ee2053928 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:56.131116 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068345f-0de0-4b08-938f-251ee2053928 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:56.137707 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 230/452] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Sep 14 01:53:55 2022] GET /identity/v3/role_assignments?group.id=2e14b4b0fee94205981b90d7ddedbc9e&scope.project.id=60ea1e9a961b4e3b99b9fe44ac610808&role.id=e842ac50986946a4971a51a06a39e16b => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:56.156168 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:56.195575 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b37265-2764-4dca-b158-f8821c50e4b4 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:56.234433 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fc971-ff9f-4f76-bb9e-957ad9e41af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:56.235066 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fc971-ff9f-4f76-bb9e-957ad9e41af8 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:56.235341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fc971-ff9f-4f76-bb9e-957ad9e41af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:56.235612 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fc971-ff9f-4f76-bb9e-957ad9e41af8 None admin] PATH_INFO: `/v3/projects/2ca72c26a2124532970b02d36e64c3f9` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:56.236780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796fc971-ff9f-4f76-bb9e-957ad9e41af8 None admin] RBAC: Authorizing `identity:get_project(project_id=2ca72c26a2124532970b02d36e64c3f9)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:56.238838 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796fc971-ff9f-4f76-bb9e-957ad9e41af8 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:56.240516 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 223/453] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:53:56 2022] GET /identity/v3/projects/2ca72c26a2124532970b02d36e64c3f9 => generated 344 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:53:56.246268 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068345f-0de0-4b08-938f-251ee2053928 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:56.359348 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068345f-0de0-4b08-938f-251ee2053928 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:56.399436 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a0841-72d6-4449-94a5-17679f157a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:56.400105 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a0841-72d6-4449-94a5-17679f157a91 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:56.400385 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a0841-72d6-4449-94a5-17679f157a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:56.400654 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a0841-72d6-4449-94a5-17679f157a91 None admin] PATH_INFO: `/v3/groups/c4e92f5e40544357b755938ebc5b8a05` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:56.401766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a0841-72d6-4449-94a5-17679f157a91 None admin] RBAC: Authorizing `identity:get_group(group_id=c4e92f5e40544357b755938ebc5b8a05)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:56.403253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a0841-72d6-4449-94a5-17679f157a91 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:56.404823 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 231/454] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 14 01:53:56 2022] GET /identity/v3/groups/c4e92f5e40544357b755938ebc5b8a05 => generated 229 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:56.411212 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fc971-ff9f-4f76-bb9e-957ad9e41af8 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:56.510296 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fc971-ff9f-4f76-bb9e-957ad9e41af8 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:56.544504 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e410f-d9ae-4552-8037-128a2a860b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:56.545105 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e410f-d9ae-4552-8037-128a2a860b64 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:56.545380 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e410f-d9ae-4552-8037-128a2a860b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:56.545647 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e410f-d9ae-4552-8037-128a2a860b64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:56.546390 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e410f-d9ae-4552-8037-128a2a860b64 None admin] RBAC: Adding query filter params (group.id=c4e92f5e40544357b755938ebc5b8a05, role.id=aa420c2fb1ab4bb3b05b004a4d89779a, scope.project.id=2ca72c26a2124532970b02d36e64c3f9) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:53:56.546694 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e410f-d9ae-4552-8037-128a2a860b64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:56.548108 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e410f-d9ae-4552-8037-128a2a860b64 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:56.554265 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 224/455] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Sep 14 01:53:56 2022] GET /identity/v3/role_assignments?group.id=c4e92f5e40544357b755938ebc5b8a05&scope.project.id=2ca72c26a2124532970b02d36e64c3f9&role.id=aa420c2fb1ab4bb3b05b004a4d89779a => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:53:59.126086 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d88f61-f324-4934-a9b0-4632ca7fbe9c None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:59.126086 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d88f61-f324-4934-a9b0-4632ca7fbe9c None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:59.126086 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d88f61-f324-4934-a9b0-4632ca7fbe9c None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:59.127151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 232/456] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:59.134270 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af7896-59b0-418b-be7a-23cc663cccd0 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:59.134793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af7896-59b0-418b-be7a-23cc663cccd0 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:59.134793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af7896-59b0-418b-be7a-23cc663cccd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:59.146400 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:59.159557 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-f5af7896-59b0-418b-be7a-23cc663cccd0 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:59.161095 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-f5af7896-59b0-418b-be7a-23cc663cccd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:59.256186 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 225/457] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:59.265407 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c1e22-bd51-4d70-b931-30f1d7df1e0d None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:59.265994 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c1e22-bd51-4d70-b931-30f1d7df1e0d None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:59.266468 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c1e22-bd51-4d70-b931-30f1d7df1e0d None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:59.268255 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 233/458] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:59.275905 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbd37f-6adf-4190-a600-467ae03b821a None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:59.276195 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbd37f-6adf-4190-a600-467ae03b821a None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:59.276472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbd37f-6adf-4190-a600-467ae03b821a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:59.302736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-38cbd37f-6adf-4190-a600-467ae03b821a None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:59.303951 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-38cbd37f-6adf-4190-a600-467ae03b821a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:59.389841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 226/459] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:59.498691 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0514265-2011-469d-9fd8-bf593f425663 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:59.499333 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0514265-2011-469d-9fd8-bf593f425663 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:59.499837 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0514265-2011-469d-9fd8-bf593f425663 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:59.501624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 234/460] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:53:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:53:59.507511 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e64eac-0ed4-486e-a4e4-9bafed851cd2 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:59.507800 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e64eac-0ed4-486e-a4e4-9bafed851cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:59.508076 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e64eac-0ed4-486e-a4e4-9bafed851cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:59.532091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-77e64eac-0ed4-486e-a4e4-9bafed851cd2 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:53:59.533278 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-77e64eac-0ed4-486e-a4e4-9bafed851cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:53:59.605507 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:59.619672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 227/461] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:53:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:53:59.655084 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0514265-2011-469d-9fd8-bf593f425663 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:59.666446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:53:59.734070 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e64eac-0ed4-486e-a4e4-9bafed851cd2 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:59.817652 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0514265-2011-469d-9fd8-bf593f425663 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:59.878914 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e64eac-0ed4-486e-a4e4-9bafed851cd2 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:53:59.883497 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469acf2-c83b-456c-b786-b3872bef729b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:59.884133 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469acf2-c83b-456c-b786-b3872bef729b None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:59.884411 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469acf2-c83b-456c-b786-b3872bef729b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:59.884682 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469acf2-c83b-456c-b786-b3872bef729b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:59.908144 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7469acf2-c83b-456c-b786-b3872bef729b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:59.910543 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7469acf2-c83b-456c-b786-b3872bef729b None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:59.930619 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bc5c3-7ce9-45d2-b904-dda82855a8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:53:59.931477 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bc5c3-7ce9-45d2-b904-dda82855a8fe None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:53:59.931641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-7469acf2-c83b-456c-b786-b3872bef729b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:53:59.931641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:59.931641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:59.931641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:59.931641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:59.931641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:59.931641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:59.931641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:59.931641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:59.931641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:59.931641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:59.931641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:59.931641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:53:59.931641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:53:59.931641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:53:59.931641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:53:59.932664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:59.932664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:59.932664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:59.932664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:59.932664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:59.932664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:59.932664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:59.932664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:59.932664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:59.932664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:59.932664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:59.932664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:59.932664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:59.932664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:59.932664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:59.932664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:59.932664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:53:59.933624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:53:59.933624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:53:59.933624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:53:59.933624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 235/462] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:53:59.933869 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469acf2-c83b-456c-b786-b3872bef729b None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:53:59.934325 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bc5c3-7ce9-45d2-b904-dda82855a8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:53:59.934715 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bc5c3-7ce9-45d2-b904-dda82855a8fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:53:59.951968 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bc5c3-7ce9-45d2-b904-dda82855a8fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:53:59.954108 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bc5c3-7ce9-45d2-b904-dda82855a8fe None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:53:59.973671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-1c7bc5c3-7ce9-45d2-b904-dda82855a8fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:53:59.973671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:53:59.973671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:53:59.973671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:53:59.973671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:53:59.973671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:53:59.973671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:53:59.973671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:53:59.973671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:53:59.973671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:53:59.973671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:53:59.973671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:53:59.973671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:53:59.973671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:53:59.973671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:53:59.973671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:53:59.974815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:53:59.974815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:53:59.974815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:53:59.974815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:53:59.974815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:53:59.974815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:53:59.974815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:53:59.974815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:53:59.974815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:53:59.974815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:53:59.974815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:53:59.974815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:53:59.974815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:53:59.974815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:53:59.974815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:53:59.974815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:53:59.974815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:53:59.975935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:53:59.975935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:53:59.975935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:53:59.978023 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 228/463] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:53:59.978998 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bc5c3-7ce9-45d2-b904-dda82855a8fe None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:00.006847 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:00.037145 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:00.058301 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469acf2-c83b-456c-b786-b3872bef729b None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:00.096649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bc5c3-7ce9-45d2-b904-dda82855a8fe None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:00.099290 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd88067-b438-481b-86ec-a28b491ad019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:00.100849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd88067-b438-481b-86ec-a28b491ad019 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:00.101293 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd88067-b438-481b-86ec-a28b491ad019 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:00.101710 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd88067-b438-481b-86ec-a28b491ad019 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:00.115588 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd88067-b438-481b-86ec-a28b491ad019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:00.117651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd88067-b438-481b-86ec-a28b491ad019 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:00.135397 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf1bb9-439c-4411-bb31-320ebba6043f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:00.136648 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf1bb9-439c-4411-bb31-320ebba6043f None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:00.137074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf1bb9-439c-4411-bb31-320ebba6043f None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:00.137192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-1dd88067-b438-481b-86ec-a28b491ad019 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:00.137192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:00.137192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:00.137192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:00.137192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:00.137192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:00.137192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:00.137192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:00.137192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:00.137192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:00.137192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:00.137192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:00.137192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:00.137192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:00.137192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:00.137192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:00.138478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:00.138478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:00.138478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:00.138478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:00.138478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:00.138478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:00.138478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:00.138478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:00.138478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:00.138478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:00.138478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:00.138478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:00.138478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:00.138478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:00.138478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:00.138478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:00.138478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:00.139351 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:00.139351 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:00.139351 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:00.140298 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf1bb9-439c-4411-bb31-320ebba6043f None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:00.141227 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf1bb9-439c-4411-bb31-320ebba6043f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:00.141718 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf1bb9-439c-4411-bb31-320ebba6043f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:00.143292 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 236/464] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:00.144503 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf1bb9-439c-4411-bb31-320ebba6043f None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:00.144820 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd88067-b438-481b-86ec-a28b491ad019 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:00.164639 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 229/465] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:00.165591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf1bb9-439c-4411-bb31-320ebba6043f None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:00.261054 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd88067-b438-481b-86ec-a28b491ad019 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:00.274771 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:00.295293 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf1bb9-439c-4411-bb31-320ebba6043f None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:00.297575 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:00.299853 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f1292-2127-4f98-8cf9-c5404c82dfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:00.301350 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f1292-2127-4f98-8cf9-c5404c82dfa3 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:00.301774 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f1292-2127-4f98-8cf9-c5404c82dfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:00.302286 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f1292-2127-4f98-8cf9-c5404c82dfa3 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:00.303064 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f1292-2127-4f98-8cf9-c5404c82dfa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:00.303766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f1292-2127-4f98-8cf9-c5404c82dfa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:00.305284 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f1292-2127-4f98-8cf9-c5404c82dfa3 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:00.322460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 237/466] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:00.323569 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f1292-2127-4f98-8cf9-c5404c82dfa3 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:00.330965 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8a09c-c323-409a-8f7a-9ef21544af2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:00.331610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8a09c-c323-409a-8f7a-9ef21544af2b None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:00.332196 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8a09c-c323-409a-8f7a-9ef21544af2b None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:00.332303 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8a09c-c323-409a-8f7a-9ef21544af2b None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:00.332914 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8a09c-c323-409a-8f7a-9ef21544af2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:00.333215 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8a09c-c323-409a-8f7a-9ef21544af2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:00.334623 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8a09c-c323-409a-8f7a-9ef21544af2b None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:00.350644 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 230/467] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:00.350644 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9707715-298b-4ce1-916d-a9ed210ff8de None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:00.350644 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9707715-298b-4ce1-916d-a9ed210ff8de None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:00.350644 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9707715-298b-4ce1-916d-a9ed210ff8de None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:00.350644 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 231/468] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:00.350644 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9707715-298b-4ce1-916d-a9ed210ff8de None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:00.443285 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f1292-2127-4f98-8cf9-c5404c82dfa3 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:00.457291 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9707715-298b-4ce1-916d-a9ed210ff8de None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:00.477865 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309aed06-0771-44bd-bfe2-cd18da46dbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:00.478505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309aed06-0771-44bd-bfe2-cd18da46dbfe None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:00.479212 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309aed06-0771-44bd-bfe2-cd18da46dbfe None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:00.479494 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309aed06-0771-44bd-bfe2-cd18da46dbfe None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:00.480223 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309aed06-0771-44bd-bfe2-cd18da46dbfe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:00.481581 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309aed06-0771-44bd-bfe2-cd18da46dbfe None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:00.492904 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06408d81-5184-4af6-8b69-91b9d320af87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:00.493521 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06408d81-5184-4af6-8b69-91b9d320af87 None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:00.493800 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06408d81-5184-4af6-8b69-91b9d320af87 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:00.494096 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06408d81-5184-4af6-8b69-91b9d320af87 None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:00.495036 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06408d81-5184-4af6-8b69-91b9d320af87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:00.496352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06408d81-5184-4af6-8b69-91b9d320af87 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:00.510342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 238/469] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 14 01:54:00 2022] POST /identity/v3/users => generated 246 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:54:00.510342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309aed06-0771-44bd-bfe2-cd18da46dbfe None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:00.532031 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 232/470] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 14 01:54:00 2022] POST /identity/v3/users => generated 248 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:54:00.533554 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05734b0-f41f-4982-acd2-185dfc75f4ee None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:00.533976 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05734b0-f41f-4982-acd2-185dfc75f4ee None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:00.534373 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05734b0-f41f-4982-acd2-185dfc75f4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:00.565534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-f05734b0-f41f-4982-acd2-185dfc75f4ee None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:00.568676 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-f05734b0-f41f-4982-acd2-185dfc75f4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:00.608899 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:00.616343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:00.749272 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 233/471] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:54:00.754537 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309aed06-0771-44bd-bfe2-cd18da46dbfe None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:00.790929 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efdc4b-95b3-4f89-ba7a-40a24d4da84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:00.791866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efdc4b-95b3-4f89-ba7a-40a24d4da84f None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:00.792338 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efdc4b-95b3-4f89-ba7a-40a24d4da84f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:00.792821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efdc4b-95b3-4f89-ba7a-40a24d4da84f None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:00.793664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efdc4b-95b3-4f89-ba7a-40a24d4da84f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:00.795305 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efdc4b-95b3-4f89-ba7a-40a24d4da84f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:00.814682 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ba01e-3bb7-49d3-b4db-04fb5184cbbe None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:00.815272 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ba01e-3bb7-49d3-b4db-04fb5184cbbe None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:00.816518 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ba01e-3bb7-49d3-b4db-04fb5184cbbe None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:00.819450 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 234/472] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:54:00.825039 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 239/473] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 14 01:54:00 2022] POST /identity/v3/users => generated 248 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:54:00.826856 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd440f-61dd-4079-9a0b-a39fefcd8c63 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:00.827323 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd440f-61dd-4079-9a0b-a39fefcd8c63 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:00.827766 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd440f-61dd-4079-9a0b-a39fefcd8c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:00.848640 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-edcd440f-61dd-4079-9a0b-a39fefcd8c63 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:00.850176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-edcd440f-61dd-4079-9a0b-a39fefcd8c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:00.944935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 235/474] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:54:01.072347 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efdc4b-95b3-4f89-ba7a-40a24d4da84f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:01.172700 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:01.193802 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efdc4b-95b3-4f89-ba7a-40a24d4da84f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:01.230318 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438c663-edbd-4e9c-b7ec-8b221281b6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:01.230964 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438c663-edbd-4e9c-b7ec-8b221281b6fe None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:01.231319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438c663-edbd-4e9c-b7ec-8b221281b6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:01.231601 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438c663-edbd-4e9c-b7ec-8b221281b6fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:01.237247 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd440f-61dd-4079-9a0b-a39fefcd8c63 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:01.244134 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438c663-edbd-4e9c-b7ec-8b221281b6fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:01.245685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438c663-edbd-4e9c-b7ec-8b221281b6fe None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:01.272943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-0438c663-edbd-4e9c-b7ec-8b221281b6fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:01.272943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:01.272943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:01.272943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:01.272943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:01.272943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:01.272943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:01.272943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:01.272943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:01.272943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:01.272943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:01.272943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:01.272943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:01.272943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:01.272943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:01.272943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:01.273940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:01.273940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:01.273940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:01.273940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:01.273940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:01.273940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:01.273940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:01.273940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:01.273940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:01.273940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:01.273940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:01.273940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:01.273940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:01.273940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:01.273940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:01.273940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:01.273940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:01.275365 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:01.275365 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:01.275365 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:01.275365 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 240/475] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:01.280705 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438c663-edbd-4e9c-b7ec-8b221281b6fe None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:01.407214 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:01.439906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438c663-edbd-4e9c-b7ec-8b221281b6fe None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:01.476010 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46f09c-fe4e-4c8c-89b5-5be7036c6374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:01.476216 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd440f-61dd-4079-9a0b-a39fefcd8c63 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:01.476620 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46f09c-fe4e-4c8c-89b5-5be7036c6374 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:01.476897 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46f09c-fe4e-4c8c-89b5-5be7036c6374 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:01.477169 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46f09c-fe4e-4c8c-89b5-5be7036c6374 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:01.477914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa46f09c-fe4e-4c8c-89b5-5be7036c6374 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:01.478238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa46f09c-fe4e-4c8c-89b5-5be7036c6374 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:01.479604 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa46f09c-fe4e-4c8c-89b5-5be7036c6374 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:01.481958 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:01.493257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 241/476] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:01.502594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46f09c-fe4e-4c8c-89b5-5be7036c6374 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:01.570454 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639800d-2188-4734-8e8e-ad6bc5f18562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:01.571084 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639800d-2188-4734-8e8e-ad6bc5f18562 None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:01.571358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639800d-2188-4734-8e8e-ad6bc5f18562 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:01.571625 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639800d-2188-4734-8e8e-ad6bc5f18562 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:01.572299 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639800d-2188-4734-8e8e-ad6bc5f18562 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:01.573675 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639800d-2188-4734-8e8e-ad6bc5f18562 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:01.598843 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.sql.core [None req-d639800d-2188-4734-8e8e-ad6bc5f18562 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 14 01:54:01.598843 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 14 01:54:01.598843 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [parameters: {'id': '4df35f0cc8ec49d9bfc80df3a715f634', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 14 01:54:01.598843 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81383) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 14 01:54:01.609939 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46f09c-fe4e-4c8c-89b5-5be7036c6374 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:01.624810 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-d639800d-2188-4734-8e8e-ad6bc5f18562 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 14 01:54:01.624810 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:01.624810 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 01:54:01.624810 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 01:54:01.624810 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 01:54:01.624810 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 01:54:01.624810 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 01:54:01.624810 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 01:54:01.624810 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 01:54:01.624810 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application conn.query(q) Sep 14 01:54:01.624810 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 01:54:01.624810 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 01:54:01.624810 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 01:54:01.624810 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application result.read() Sep 14 01:54:01.624810 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 01:54:01.625739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 01:54:01.625739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 01:54:01.625739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 01:54:01.625739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 01:54:01.625739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 01:54:01.625739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 01:54:01.625739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 01:54:01.625739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 14 01:54:01.625739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:01.625739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 14 01:54:01.625739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:01.625739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:01.625739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 14 01:54:01.625739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 14 01:54:01.625739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 14 01:54:01.625739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return ref.to_dict() Sep 14 01:54:01.625739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 01:54:01.625739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application next(self.gen) Sep 14 01:54:01.626733 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 14 01:54:01.626733 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application yield resource Sep 14 01:54:01.626733 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 01:54:01.626733 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application next(self.gen) Sep 14 01:54:01.626733 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 14 01:54:01.626733 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self.session.rollback() Sep 14 01:54:01.626733 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 14 01:54:01.626733 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self.force_reraise() Sep 14 01:54:01.626733 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 14 01:54:01.626733 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise self.value Sep 14 01:54:01.626733 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 14 01:54:01.626733 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 14 01:54:01.626733 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 14 01:54:01.626733 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application session.commit() Sep 14 01:54:01.626733 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 14 01:54:01.626733 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 14 01:54:01.626733 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 14 01:54:01.627610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._prepare_impl() Sep 14 01:54:01.627610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 14 01:54:01.627610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self.session.flush() Sep 14 01:54:01.627610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 14 01:54:01.627610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._flush(objects) Sep 14 01:54:01.627610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 14 01:54:01.627610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 14 01:54:01.627610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 14 01:54:01.627610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application compat.raise_( Sep 14 01:54:01.627610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 01:54:01.627610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception Sep 14 01:54:01.627610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 14 01:54:01.627610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application flush_context.execute() Sep 14 01:54:01.627610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 14 01:54:01.627610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rec.execute(self) Sep 14 01:54:01.627610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 14 01:54:01.627610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 14 01:54:01.628566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 14 01:54:01.628566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 14 01:54:01.628566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 14 01:54:01.628566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 14 01:54:01.628566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 14 01:54:01.628566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 14 01:54:01.628566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 14 01:54:01.628566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 14 01:54:01.628566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 14 01:54:01.628566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 14 01:54:01.628566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 14 01:54:01.628566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 14 01:54:01.628566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 14 01:54:01.628566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 14 01:54:01.628566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 01:54:01.628566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception Sep 14 01:54:01.629438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 01:54:01.629438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 01:54:01.629438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 01:54:01.629438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 01:54:01.629438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 01:54:01.629438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 01:54:01.629438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 01:54:01.629438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application conn.query(q) Sep 14 01:54:01.629438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 01:54:01.629438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 01:54:01.629438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 01:54:01.629438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application result.read() Sep 14 01:54:01.629438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 01:54:01.629438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 01:54:01.629438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 01:54:01.629438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 01:54:01.631864 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 01:54:01.631864 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 01:54:01.631864 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 01:54:01.631864 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 01:54:01.631864 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 14 01:54:01.631864 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 14 01:54:01.631864 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [parameters: {'id': '4df35f0cc8ec49d9bfc80df3a715f634', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 14 01:54:01.631864 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 14 01:54:01.631864 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:01.631864 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:54:01.631864 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:01.631864 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:01.631864 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:01.631864 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:01.631864 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:01.631864 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:01.633806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:01.633806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:01.633806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:01.633806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:01.633806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:01.633806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:01.633806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 14 01:54:01.633806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 14 01:54:01.633806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:01.633806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:01.633806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 14 01:54:01.633806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 14 01:54:01.633806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 14 01:54:01.633806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 14 01:54:01.633806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 14 01:54:01.633806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:01.640575 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 236/477] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Sep 14 01:54:01 2022] POST /identity/v3/roles => generated 144 bytes in 398 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:01.642063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639800d-2188-4734-8e8e-ad6bc5f18562 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:01.651691 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f39d88-d2be-4d25-8768-dc1edd6395e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:01.652316 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f39d88-d2be-4d25-8768-dc1edd6395e6 None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:01.652738 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f39d88-d2be-4d25-8768-dc1edd6395e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:01.653130 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f39d88-d2be-4d25-8768-dc1edd6395e6 None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:01.653800 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f39d88-d2be-4d25-8768-dc1edd6395e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:01.655309 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f39d88-d2be-4d25-8768-dc1edd6395e6 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:01.692118 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 242/478] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 14 01:54:01 2022] POST /identity/v3/users => generated 249 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:54:01.768234 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639800d-2188-4734-8e8e-ad6bc5f18562 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:01.805121 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c82712-24e9-412d-b577-447c94866b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:01.805786 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c82712-24e9-412d-b577-447c94866b1e None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:01.806094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c82712-24e9-412d-b577-447c94866b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:01.806369 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c82712-24e9-412d-b577-447c94866b1e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:01.818636 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c82712-24e9-412d-b577-447c94866b1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:01.824038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-b5c82712-24e9-412d-b577-447c94866b1e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:54:01.824038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:01.824038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:01.824038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:01.824038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:01.824038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:01.824038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:01.824038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:01.824038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:01.824038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:01.824038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:01.824038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:01.824038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:01.824038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:01.824038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:01.824038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:01.826035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:01.826035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:01.826035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:01.826035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:01.826035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:01.826035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:01.826035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:01.826035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:01.826035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:01.826035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:01.826035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:01.826035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:01.826035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:01.826035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:01.826035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:01.826035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:01.827559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:01.827559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:01.827559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:01.827559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:01.827559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:01.827559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:01.827559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:01.827559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:01.827559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:01.827559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:01.827559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:54:01.827559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:01.827559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 237/479] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:54:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:01.831366 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f39d88-d2be-4d25-8768-dc1edd6395e6 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:01.925403 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:01.981118 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f39d88-d2be-4d25-8768-dc1edd6395e6 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:02.018494 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c2a47-538b-4585-b269-9dc623976b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:02.019159 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c2a47-538b-4585-b269-9dc623976b62 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:02.019441 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c2a47-538b-4585-b269-9dc623976b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:02.019713 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c2a47-538b-4585-b269-9dc623976b62 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:02.020371 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c2a47-538b-4585-b269-9dc623976b62 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:02.020673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c2a47-538b-4585-b269-9dc623976b62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:02.022177 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c2a47-538b-4585-b269-9dc623976b62 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:02.030952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 243/480] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:03.188533 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e41f7-a8b7-4b9f-b789-2cacb0d4a3a2 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:03.188533 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e41f7-a8b7-4b9f-b789-2cacb0d4a3a2 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:03.188533 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e41f7-a8b7-4b9f-b789-2cacb0d4a3a2 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:03.188533 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 238/481] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:03.197859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db864f2-557b-4990-8f71-a15d3c6a7847 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:03.197859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db864f2-557b-4990-8f71-a15d3c6a7847 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:03.197859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db864f2-557b-4990-8f71-a15d3c6a7847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:03.223480 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-1db864f2-557b-4990-8f71-a15d3c6a7847 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:03.225139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-1db864f2-557b-4990-8f71-a15d3c6a7847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:03.321255 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 244/482] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:54:03.364529 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ccaae-7425-43c2-8dc8-d183ed1342e4 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:03.365151 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ccaae-7425-43c2-8dc8-d183ed1342e4 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:03.365625 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ccaae-7425-43c2-8dc8-d183ed1342e4 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:03.367475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 239/483] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:03.372257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fc16b-2f25-4cbc-9297-35d156e3dc95 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:03.372547 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fc16b-2f25-4cbc-9297-35d156e3dc95 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:03.372821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fc16b-2f25-4cbc-9297-35d156e3dc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:03.394239 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-976fc16b-2f25-4cbc-9297-35d156e3dc95 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:03.395440 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-976fc16b-2f25-4cbc-9297-35d156e3dc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:03.448179 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:03.488525 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 245/484] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:54:03.623431 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b3e71-dada-40e6-ae5b-11752358fdc3 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:03.623431 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b3e71-dada-40e6-ae5b-11752358fdc3 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:03.623431 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b3e71-dada-40e6-ae5b-11752358fdc3 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:03.624196 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 240/485] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:03.629529 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cfcfa-72ba-4034-a99e-fc65c210e70a None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:03.630049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cfcfa-72ba-4034-a99e-fc65c210e70a None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:03.630522 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cfcfa-72ba-4034-a99e-fc65c210e70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:03.649694 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-ed1cfcfa-72ba-4034-a99e-fc65c210e70a None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:03.651149 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-ed1cfcfa-72ba-4034-a99e-fc65c210e70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:03.729044 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b3e71-dada-40e6-ae5b-11752358fdc3 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:03.733188 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:03.747592 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 246/486] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:54:03.806417 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cfcfa-72ba-4034-a99e-fc65c210e70a None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:03.902795 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b3e71-dada-40e6-ae5b-11752358fdc3 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:03.962405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb412d-1e7c-4d3a-80ea-b2fbb535824b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:03.963421 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb412d-1e7c-4d3a-80ea-b2fbb535824b None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:03.963900 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb412d-1e7c-4d3a-80ea-b2fbb535824b None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:03.964379 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb412d-1e7c-4d3a-80ea-b2fbb535824b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:03.973785 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cfcfa-72ba-4034-a99e-fc65c210e70a None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:03.986413 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb412d-1e7c-4d3a-80ea-b2fbb535824b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:03.992013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-2cbb412d-1e7c-4d3a-80ea-b2fbb535824b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:03.992013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:03.992013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:03.992013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:03.992013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:03.992013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:03.992013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:03.992013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:03.992013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:03.992013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:03.992013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:03.992013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:03.992013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:03.992013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:03.992013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:03.992013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:03.992912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:03.992912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:03.992912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:03.992912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:03.992912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:03.992912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:03.992912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:03.992912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:03.992912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:03.992912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:03.992912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:03.992912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:03.992912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:03.992912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:03.992912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:03.992912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:03.993721 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:03.993721 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:03.993721 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:03.993721 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:03.993721 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:03.993721 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:03.993721 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:03.993721 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:03.993721 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:03.993721 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:03.993721 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:03.993721 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:03.996401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 241/487] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:54:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:03.996814 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:04.004249 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb412d-1e7c-4d3a-80ea-b2fbb535824b None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:04.045203 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6d6df-20f9-4931-9004-d16925e1dd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:04.046268 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6d6df-20f9-4931-9004-d16925e1dd5a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:04.046749 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6d6df-20f9-4931-9004-d16925e1dd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:04.047238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6d6df-20f9-4931-9004-d16925e1dd5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:04.058845 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef6d6df-20f9-4931-9004-d16925e1dd5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:04.063940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-cef6d6df-20f9-4931-9004-d16925e1dd5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:04.063940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:04.063940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:04.063940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:04.063940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:04.063940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:04.063940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:04.063940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:04.063940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:04.063940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:04.063940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:04.063940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:04.063940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:04.063940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:04.063940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:04.063940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:04.064859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:04.064859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:04.064859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:04.064859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:04.064859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:04.064859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:04.064859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:04.064859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:04.064859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:04.064859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:04.064859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:04.064859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:04.064859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:04.064859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:04.064859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:04.064859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:04.066246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:04.066246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:04.066246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:04.066246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:04.066246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:04.066246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:04.066246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:04.066246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:04.066246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:04.066246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:04.066246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:04.066246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:04.068492 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 247/488] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:54:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:04.074184 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6d6df-20f9-4931-9004-d16925e1dd5a None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:04.095360 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:04.149830 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb412d-1e7c-4d3a-80ea-b2fbb535824b None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:04.190152 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fd31f-aeec-449b-b4bc-a23f9e41afaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:04.192059 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fd31f-aeec-449b-b4bc-a23f9e41afaa None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:04.192564 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fd31f-aeec-449b-b4bc-a23f9e41afaa None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:04.193041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fd31f-aeec-449b-b4bc-a23f9e41afaa None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:04.193671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6d6df-20f9-4931-9004-d16925e1dd5a None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:04.194129 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064fd31f-aeec-449b-b4bc-a23f9e41afaa None admin] RBAC: Adding query filter params (name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:04.194628 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064fd31f-aeec-449b-b4bc-a23f9e41afaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:04.196241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064fd31f-aeec-449b-b4bc-a23f9e41afaa None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:04.205152 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 242/489] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 14 01:54:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:04.206496 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fd31f-aeec-449b-b4bc-a23f9e41afaa None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:04.233301 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078415c-5ac7-40a2-becd-281b514bbd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:04.233976 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078415c-5ac7-40a2-becd-281b514bbd8e None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:04.234236 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078415c-5ac7-40a2-becd-281b514bbd8e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:04.234511 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078415c-5ac7-40a2-becd-281b514bbd8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:04.245647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078415c-5ac7-40a2-becd-281b514bbd8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:04.250374 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-8078415c-5ac7-40a2-becd-281b514bbd8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:04.250374 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:04.250374 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:04.250374 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:04.250374 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:04.250374 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:04.250374 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:04.250374 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:04.250374 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:04.250374 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:04.250374 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:04.250374 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:04.250374 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:04.250374 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:04.250374 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:04.250374 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:04.251291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:04.251291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:04.251291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:04.251291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:04.251291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:04.251291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:04.251291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:04.251291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:04.251291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:04.251291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:04.251291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:04.251291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:04.251291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:04.251291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:04.251291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:04.251291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:04.252472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:04.252472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:04.252472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:04.252472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:04.252472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:04.252472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:04.252472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:04.252472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:04.252472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:04.252472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:04.252472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:04.252472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:04.252472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 248/490] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:54:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:04.252472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078415c-5ac7-40a2-becd-281b514bbd8e None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:04.282625 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:04.319605 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fd31f-aeec-449b-b4bc-a23f9e41afaa None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:04.347558 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:04.357401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def7ca7-b84f-4b1b-a108-aa28669f5f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:04.358229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def7ca7-b84f-4b1b-a108-aa28669f5f53 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:04.358522 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def7ca7-b84f-4b1b-a108-aa28669f5f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:04.358798 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def7ca7-b84f-4b1b-a108-aa28669f5f53 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:04.359475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def7ca7-b84f-4b1b-a108-aa28669f5f53 None admin] RBAC: Adding query filter params (name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:04.359784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def7ca7-b84f-4b1b-a108-aa28669f5f53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:04.361258 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def7ca7-b84f-4b1b-a108-aa28669f5f53 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:04.368041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078415c-5ac7-40a2-becd-281b514bbd8e None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:04.373169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 243/491] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 14 01:54:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:04.378813 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def7ca7-b84f-4b1b-a108-aa28669f5f53 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:04.409379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f10d6-4b5e-44f1-bfdf-ac7bb8b43114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:04.410420 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f10d6-4b5e-44f1-bfdf-ac7bb8b43114 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:04.410897 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f10d6-4b5e-44f1-bfdf-ac7bb8b43114 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:04.411382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f10d6-4b5e-44f1-bfdf-ac7bb8b43114 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:04.422877 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f10d6-4b5e-44f1-bfdf-ac7bb8b43114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:04.424629 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f10d6-4b5e-44f1-bfdf-ac7bb8b43114 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:04.440011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-761f10d6-4b5e-44f1-bfdf-ac7bb8b43114 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:04.440011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:04.440011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:04.440011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:04.440011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:04.440011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:04.440011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:04.440011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:04.440011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:04.440011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:04.440011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:04.440011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:04.440011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:04.440011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:04.440011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:04.440011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:04.441036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:04.441036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:04.441036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:04.441036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:04.441036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:04.441036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:04.441036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:04.441036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:04.441036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:04.441036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:04.441036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:04.441036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:04.441036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:04.441036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:04.441036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:04.441036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:04.441036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:04.441909 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:04.441909 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:04.441909 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:04.443662 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 249/492] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:04.444647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f10d6-4b5e-44f1-bfdf-ac7bb8b43114 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:04.492667 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def7ca7-b84f-4b1b-a108-aa28669f5f53 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:04.526252 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:04.528406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b822f9-84ef-4851-8f47-829fb2605550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:04.529176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b822f9-84ef-4851-8f47-829fb2605550 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:04.529689 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b822f9-84ef-4851-8f47-829fb2605550 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:04.529956 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b822f9-84ef-4851-8f47-829fb2605550 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:04.530637 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b822f9-84ef-4851-8f47-829fb2605550 None admin] RBAC: Adding query filter params (name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:04.530935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b822f9-84ef-4851-8f47-829fb2605550 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:04.532287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b822f9-84ef-4851-8f47-829fb2605550 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:04.540773 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 244/493] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 14 01:54:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:04.541722 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b822f9-84ef-4851-8f47-829fb2605550 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:04.554000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f10d6-4b5e-44f1-bfdf-ac7bb8b43114 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:04.587881 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c58734-0f83-4e79-a7b6-3b526c90f495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:04.588507 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c58734-0f83-4e79-a7b6-3b526c90f495 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:04.588786 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c58734-0f83-4e79-a7b6-3b526c90f495 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:04.589058 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c58734-0f83-4e79-a7b6-3b526c90f495 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:04.594574 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:04.600653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c58734-0f83-4e79-a7b6-3b526c90f495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:04.602151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c58734-0f83-4e79-a7b6-3b526c90f495 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:04.616575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-b4c58734-0f83-4e79-a7b6-3b526c90f495 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:04.616575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:04.616575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:04.616575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:04.616575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:04.616575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:04.616575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:04.616575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:04.616575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:04.616575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:04.616575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:04.616575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:04.616575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:04.616575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:04.616575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:04.616575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:04.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:04.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:04.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:04.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:04.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:04.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:04.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:04.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:04.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:04.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:04.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:04.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:04.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:04.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:04.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:04.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:04.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:04.618384 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:04.618384 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:04.618384 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:04.618384 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 250/494] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:04.619052 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec4156-0489-4a5e-85a7-b07ad40253cf None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:04.619052 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec4156-0489-4a5e-85a7-b07ad40253cf None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:04.619313 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec4156-0489-4a5e-85a7-b07ad40253cf None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:04.620682 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 251/495] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:04.625207 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec4156-0489-4a5e-85a7-b07ad40253cf None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:04.653950 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b822f9-84ef-4851-8f47-829fb2605550 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:04.691573 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36c78f-552f-4dbf-83c0-b1feaf6f57a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:04.692147 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36c78f-552f-4dbf-83c0-b1feaf6f57a4 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:04.692429 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36c78f-552f-4dbf-83c0-b1feaf6f57a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:04.692701 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36c78f-552f-4dbf-83c0-b1feaf6f57a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:04.693399 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36c78f-552f-4dbf-83c0-b1feaf6f57a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:04.693705 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36c78f-552f-4dbf-83c0-b1feaf6f57a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:04.695123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36c78f-552f-4dbf-83c0-b1feaf6f57a4 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:04.711522 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 245/496] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:04.712242 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36c78f-552f-4dbf-83c0-b1feaf6f57a4 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:04.732687 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec4156-0489-4a5e-85a7-b07ad40253cf None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:04.771905 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:04.785067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce521c-444d-4087-a267-377adc3c291a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:04.786061 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce521c-444d-4087-a267-377adc3c291a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:04.786562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce521c-444d-4087-a267-377adc3c291a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:04.787049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce521c-444d-4087-a267-377adc3c291a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:04.804766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce521c-444d-4087-a267-377adc3c291a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:04.807557 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce521c-444d-4087-a267-377adc3c291a None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:04.828089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-7cce521c-444d-4087-a267-377adc3c291a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:04.828089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:04.828089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:04.828089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:04.828089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:04.828089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:04.828089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:04.828089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:04.828089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:04.828089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:04.828089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:04.828089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:04.828089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:04.828089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:04.828089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:04.828089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:04.829169 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:04.829169 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:04.829169 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:04.829169 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:04.829169 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:04.829169 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:04.829169 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:04.829169 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:04.829169 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:04.829169 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:04.829169 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:04.829169 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:04.829169 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:04.829169 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:04.829169 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:04.829169 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:04.829169 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:04.830237 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:04.830237 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:04.830237 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:04.830237 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 252/497] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:04.830639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288c53d-ec83-48b0-bc31-da8a8a1e1f9b None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:04.830922 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288c53d-ec83-48b0-bc31-da8a8a1e1f9b None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:04.831731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288c53d-ec83-48b0-bc31-da8a8a1e1f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:04.833080 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36c78f-552f-4dbf-83c0-b1feaf6f57a4 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:04.845167 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:04.851137 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-0288c53d-ec83-48b0-bc31-da8a8a1e1f9b None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:04.852316 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-0288c53d-ec83-48b0-bc31-da8a8a1e1f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:04.870766 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549ff34-94e6-4ea7-b07f-6e2310fff8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:04.871366 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549ff34-94e6-4ea7-b07f-6e2310fff8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:04.871757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549ff34-94e6-4ea7-b07f-6e2310fff8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:04.872032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549ff34-94e6-4ea7-b07f-6e2310fff8d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:04.873012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549ff34-94e6-4ea7-b07f-6e2310fff8d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:04.873012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549ff34-94e6-4ea7-b07f-6e2310fff8d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:04.874362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549ff34-94e6-4ea7-b07f-6e2310fff8d4 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:04.887542 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 246/498] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:04.888335 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549ff34-94e6-4ea7-b07f-6e2310fff8d4 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:04.923821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 253/499] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:54:04.924656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288c53d-ec83-48b0-bc31-da8a8a1e1f9b None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:04.996907 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549ff34-94e6-4ea7-b07f-6e2310fff8d4 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:05.027878 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288c53d-ec83-48b0-bc31-da8a8a1e1f9b None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:05.031315 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c40aad-112f-462f-9684-288a6943c0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:05.031936 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c40aad-112f-462f-9684-288a6943c0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:05.032211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c40aad-112f-462f-9684-288a6943c0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:05.032480 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c40aad-112f-462f-9684-288a6943c0c2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:05.039737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:05.041296 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c40aad-112f-462f-9684-288a6943c0c2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:05.042725 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c40aad-112f-462f-9684-288a6943c0c2 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:05.055646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-84c40aad-112f-462f-9684-288a6943c0c2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:54:05.055646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:05.055646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:05.055646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:05.055646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:05.055646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:05.055646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:05.055646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:05.055646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:05.055646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:05.055646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:05.055646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:05.055646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:05.055646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:05.055646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:05.055646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:05.056816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:05.056816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:05.056816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:05.056816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:05.056816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:05.056816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:05.056816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:05.056816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:05.056816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:05.056816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:05.056816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:05.056816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:05.056816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:05.056816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:05.056816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:05.056816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:05.056816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:05.056816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:05.057899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:05.057899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:05.057899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:05.057899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:05.057899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:05.057899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:05.057899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:05.057899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:05.057899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:54:05.057899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:05.057899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 247/500] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Sep 14 01:54:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:05.058519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85296ae3-7ec5-484a-876d-5705f45aea02 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:05.058519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85296ae3-7ec5-484a-876d-5705f45aea02 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:05.058708 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85296ae3-7ec5-484a-876d-5705f45aea02 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:05.060072 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 248/501] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:05.060572 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85296ae3-7ec5-484a-876d-5705f45aea02 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:05.073745 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef29ea-9c68-4202-b3da-c89724fe5cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:05.074421 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef29ea-9c68-4202-b3da-c89724fe5cae None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:05.074701 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef29ea-9c68-4202-b3da-c89724fe5cae None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:05.074976 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef29ea-9c68-4202-b3da-c89724fe5cae None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:05.075611 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef29ea-9c68-4202-b3da-c89724fe5cae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:05.075914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef29ea-9c68-4202-b3da-c89724fe5cae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:05.077354 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef29ea-9c68-4202-b3da-c89724fe5cae None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:05.089307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 254/502] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:05.093903 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef29ea-9c68-4202-b3da-c89724fe5cae None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:05.102527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:05.185143 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85296ae3-7ec5-484a-876d-5705f45aea02 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:05.195154 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef29ea-9c68-4202-b3da-c89724fe5cae None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:05.221998 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8bdc92-8522-432a-b70f-589fde20d9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:05.223077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bdc92-8522-432a-b70f-589fde20d9ef None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:05.223551 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bdc92-8522-432a-b70f-589fde20d9ef None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:05.224056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bdc92-8522-432a-b70f-589fde20d9ef None admin] PATH_INFO: `/v3/users/glance` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:05.229080 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b72e9-b8be-4c50-a3e1-8777ac974565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:05.229680 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b72e9-b8be-4c50-a3e1-8777ac974565 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:05.229955 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b72e9-b8be-4c50-a3e1-8777ac974565 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:05.230328 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b72e9-b8be-4c50-a3e1-8777ac974565 None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:05.231049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b72e9-b8be-4c50-a3e1-8777ac974565 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:05.231350 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b72e9-b8be-4c50-a3e1-8777ac974565 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:05.232510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8bdc92-8522-432a-b70f-589fde20d9ef None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:05.232773 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b72e9-b8be-4c50-a3e1-8777ac974565 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:05.233946 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8bdc92-8522-432a-b70f-589fde20d9ef None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:05.245463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-be8bdc92-8522-432a-b70f-589fde20d9ef None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 14 01:54:05.245463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:05.245463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:05.245463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:05.245463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:05.245463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:05.245463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:05.245463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:05.245463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:05.245463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:05.245463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:05.245463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:05.245463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:05.245463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:05.245463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:05.245463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:05.246485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:05.246485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:05.246485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:05.246485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:05.246485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:05.246485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:05.246485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:05.246485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:05.246485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:05.246485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:05.246485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:05.246485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:05.246485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:05.246485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:05.246485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:05.246485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:05.246485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:05.246485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:05.247421 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:05.247421 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:05.247421 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:05.247421 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:05.247421 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:05.247421 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:05.247421 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:05.247421 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:05.247421 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 14 01:54:05.247421 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:05.247421 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 249/503] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:54:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:05.247996 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be21c9-2b89-4b76-ba43-b8b18557376e None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:05.248061 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be21c9-2b89-4b76-ba43-b8b18557376e None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:05.248597 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be21c9-2b89-4b76-ba43-b8b18557376e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:05.266451 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-67be21c9-2b89-4b76-ba43-b8b18557376e None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:05.267650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-67be21c9-2b89-4b76-ba43-b8b18557376e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:05.269873 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 255/504] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Sep 14 01:54:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:05.270742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b72e9-b8be-4c50-a3e1-8777ac974565 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:05.301558 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:05.336525 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 250/505] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:54:05.337433 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be21c9-2b89-4b76-ba43-b8b18557376e None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:05.368111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:05.373136 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b72e9-b8be-4c50-a3e1-8777ac974565 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:05.415837 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3278e71-a7c7-4516-be90-256b33b0cf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:05.416463 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3278e71-a7c7-4516-be90-256b33b0cf88 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:05.416833 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3278e71-a7c7-4516-be90-256b33b0cf88 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:05.417009 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3278e71-a7c7-4516-be90-256b33b0cf88 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:05.424178 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3278e71-a7c7-4516-be90-256b33b0cf88 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:05.425673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3278e71-a7c7-4516-be90-256b33b0cf88 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:05.438830 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-b3278e71-a7c7-4516-be90-256b33b0cf88 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 01:54:05.438830 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:05.438830 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:05.438830 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:05.438830 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:05.438830 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:05.438830 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:05.438830 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:05.438830 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:05.438830 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:05.438830 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:05.438830 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:05.438830 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:05.438830 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:05.438830 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:05.438830 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:05.439742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:05.439742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:05.439742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:05.439742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:05.439742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:05.439742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:05.439742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:05.439742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:05.439742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:05.439742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:05.439742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:05.439742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:05.439742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:05.439742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:05.439742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:05.439742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:05.439742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:05.439742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:05.441207 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:05.441207 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:05.441207 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:05.441207 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:05.441207 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:05.441207 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:05.441207 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:05.441207 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:05.441207 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 01:54:05.441207 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:05.441207 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 256/506] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:54:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:05.445720 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3278e71-a7c7-4516-be90-256b33b0cf88 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:05.454615 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be21c9-2b89-4b76-ba43-b8b18557376e None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:05.496317 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bbe88-4d3c-4193-9d8a-0def81109e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:05.496948 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bbe88-4d3c-4193-9d8a-0def81109e7f None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:05.497221 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bbe88-4d3c-4193-9d8a-0def81109e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:05.497489 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bbe88-4d3c-4193-9d8a-0def81109e7f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:05.508408 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4bbe88-4d3c-4193-9d8a-0def81109e7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:05.513091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-5f4bbe88-4d3c-4193-9d8a-0def81109e7f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:05.513091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:05.513091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:05.513091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:05.513091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:05.513091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:05.513091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:05.513091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:05.513091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:05.513091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:05.513091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:05.513091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:05.513091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:05.513091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:05.513091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:05.513091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:05.513973 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:05.513973 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:05.513973 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:05.513973 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:05.513973 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:05.513973 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:05.513973 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:05.513973 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:05.513973 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:05.513973 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:05.513973 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:05.513973 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:05.513973 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:05.513973 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:05.513973 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:05.513973 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:05.514841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:05.514841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:05.514841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:05.514841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:05.514841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:05.514841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:05.514841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:05.514841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:05.514841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:05.514841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:05.514841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:05.514841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:05.514841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 251/507] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:54:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:05.515508 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bbe88-4d3c-4193-9d8a-0def81109e7f None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:05.542014 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:05.555949 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3278e71-a7c7-4516-be90-256b33b0cf88 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:05.591498 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01089aa-3a72-4cfe-b3f4-50e5160a8af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:05.592147 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01089aa-3a72-4cfe-b3f4-50e5160a8af4 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:05.592430 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01089aa-3a72-4cfe-b3f4-50e5160a8af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:05.592931 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01089aa-3a72-4cfe-b3f4-50e5160a8af4 None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:05.593398 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01089aa-3a72-4cfe-b3f4-50e5160a8af4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:05.593697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01089aa-3a72-4cfe-b3f4-50e5160a8af4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:05.595158 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01089aa-3a72-4cfe-b3f4-50e5160a8af4 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:05.599493 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:05.613015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 257/508] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Sep 14 01:54:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:05.613822 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01089aa-3a72-4cfe-b3f4-50e5160a8af4 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:05.620852 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bbe88-4d3c-4193-9d8a-0def81109e7f None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:05.655519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cce176-cefc-4822-91f9-f08ba35a2099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:05.656114 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cce176-cefc-4822-91f9-f08ba35a2099 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:05.656392 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cce176-cefc-4822-91f9-f08ba35a2099 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:05.656668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cce176-cefc-4822-91f9-f08ba35a2099 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:05.712456 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01089aa-3a72-4cfe-b3f4-50e5160a8af4 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:05.723395 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cce176-cefc-4822-91f9-f08ba35a2099 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:05.724851 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cce176-cefc-4822-91f9-f08ba35a2099 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:05.745091 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32059af9-a89e-4688-a8a7-f424da4b38da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:05.745682 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32059af9-a89e-4688-a8a7-f424da4b38da None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:05.745960 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32059af9-a89e-4688-a8a7-f424da4b38da None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:05.746260 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32059af9-a89e-4688-a8a7-f424da4b38da None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:05.746973 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32059af9-a89e-4688-a8a7-f424da4b38da None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:05.747279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32059af9-a89e-4688-a8a7-f424da4b38da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:05.748696 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32059af9-a89e-4688-a8a7-f424da4b38da None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:05.766628 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 258/509] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Sep 14 01:54:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:05.767493 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32059af9-a89e-4688-a8a7-f424da4b38da None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:05.770837 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:05.806558 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:05.826361 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 252/510] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:54:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:54:05.827168 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cce176-cefc-4822-91f9-f08ba35a2099 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:05.867644 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32059af9-a89e-4688-a8a7-f424da4b38da None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:05.901571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76b8cf-2ae1-424f-885b-7b44e905b750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:05.902199 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76b8cf-2ae1-424f-885b-7b44e905b750 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:05.902475 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76b8cf-2ae1-424f-885b-7b44e905b750 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:05.902743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76b8cf-2ae1-424f-885b-7b44e905b750 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:05.903391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76b8cf-2ae1-424f-885b-7b44e905b750 None admin] RBAC: Adding query filter params (name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:05.903690 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76b8cf-2ae1-424f-885b-7b44e905b750 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:05.905037 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76b8cf-2ae1-424f-885b-7b44e905b750 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:05.912716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 259/511] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 14 01:54:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:05.913488 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76b8cf-2ae1-424f-885b-7b44e905b750 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:05.926187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cce176-cefc-4822-91f9-f08ba35a2099 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:05.959274 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b837b-db6a-4c33-aa46-921762768d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:05.959896 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b837b-db6a-4c33-aa46-921762768d8d None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:05.960175 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b837b-db6a-4c33-aa46-921762768d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:05.960427 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b837b-db6a-4c33-aa46-921762768d8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:05.970205 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b837b-db6a-4c33-aa46-921762768d8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:05.971659 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b837b-db6a-4c33-aa46-921762768d8d None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:05.975571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:05.985925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-444b837b-db6a-4c33-aa46-921762768d8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:05.985925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:05.985925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:05.985925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:05.985925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:05.985925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:05.985925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:05.985925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:05.985925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:05.985925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:05.985925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:05.985925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:05.985925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:05.985925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:05.985925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:05.985925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:05.986963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:05.986963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:05.986963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:05.986963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:05.986963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:05.986963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:05.986963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:05.986963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:05.986963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:05.986963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:05.986963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:05.986963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:05.986963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:05.986963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:05.986963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:05.986963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:05.986963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:05.988383 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:05.988383 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:05.988383 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:05.988383 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 253/512] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:05.988383 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b837b-db6a-4c33-aa46-921762768d8d None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:06.012321 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76b8cf-2ae1-424f-885b-7b44e905b750 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:06.020031 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:06.044972 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a1b32-f5c0-4ae4-9293-ae99009117a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:06.045575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a1b32-f5c0-4ae4-9293-ae99009117a4 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:06.045851 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a1b32-f5c0-4ae4-9293-ae99009117a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:06.046156 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a1b32-f5c0-4ae4-9293-ae99009117a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:06.087821 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b837b-db6a-4c33-aa46-921762768d8d None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:06.111025 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a1b32-f5c0-4ae4-9293-ae99009117a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:06.113024 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a1b32-f5c0-4ae4-9293-ae99009117a4 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:06.121157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6479295-e3e6-4cf5-bf31-37ee76d43823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:06.121752 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6479295-e3e6-4cf5-bf31-37ee76d43823 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:06.122335 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6479295-e3e6-4cf5-bf31-37ee76d43823 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:06.122335 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6479295-e3e6-4cf5-bf31-37ee76d43823 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:06.174748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:06.186794 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6479295-e3e6-4cf5-bf31-37ee76d43823 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:06.188248 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6479295-e3e6-4cf5-bf31-37ee76d43823 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:06.210774 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 260/513] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:54:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:54:06.211580 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a1b32-f5c0-4ae4-9293-ae99009117a4 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:06.218113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:06.285350 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 254/514] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:54:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:54:06.286437 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6479295-e3e6-4cf5-bf31-37ee76d43823 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:06.311784 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a1b32-f5c0-4ae4-9293-ae99009117a4 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:06.344641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e301b73-8499-4704-9264-ca8950abf0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:06.345206 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e301b73-8499-4704-9264-ca8950abf0b0 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:06.345486 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e301b73-8499-4704-9264-ca8950abf0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:06.345759 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e301b73-8499-4704-9264-ca8950abf0b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:06.356155 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e301b73-8499-4704-9264-ca8950abf0b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:06.357617 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e301b73-8499-4704-9264-ca8950abf0b0 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:06.372034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-0e301b73-8499-4704-9264-ca8950abf0b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:06.372034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:06.372034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:06.372034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:06.372034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:06.372034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:06.372034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:06.372034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:06.372034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:06.372034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:06.372034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:06.372034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:06.372034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:06.372034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:06.372034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:06.372034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:06.372952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:06.372952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:06.372952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:06.372952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:06.372952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:06.372952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:06.372952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:06.372952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:06.372952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:06.372952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:06.372952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:06.372952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:06.372952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:06.372952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:06.372952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:06.372952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:06.372952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:06.373816 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:06.373816 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:06.373816 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:06.373816 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 261/515] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:06.374072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e301b73-8499-4704-9264-ca8950abf0b0 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:06.384499 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6479295-e3e6-4cf5-bf31-37ee76d43823 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:06.395115 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:06.415084 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:06.421322 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41c821-9dbd-473f-97d5-efb56c410e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:06.421945 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41c821-9dbd-473f-97d5-efb56c410e7c None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:06.422256 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41c821-9dbd-473f-97d5-efb56c410e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:06.422529 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41c821-9dbd-473f-97d5-efb56c410e7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:06.432305 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41c821-9dbd-473f-97d5-efb56c410e7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:06.433741 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41c821-9dbd-473f-97d5-efb56c410e7c None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:06.448375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-ca41c821-9dbd-473f-97d5-efb56c410e7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:06.448375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:06.448375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:06.448375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:06.448375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:06.448375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:06.448375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:06.448375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:06.448375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:06.448375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:06.448375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:06.448375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:06.448375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:06.448375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:06.448375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:06.448375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:06.449454 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:06.449454 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:06.449454 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:06.449454 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:06.449454 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:06.449454 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:06.449454 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:06.449454 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:06.449454 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:06.449454 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:06.449454 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:06.449454 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:06.449454 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:06.449454 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:06.449454 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:06.449454 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:06.449454 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:06.450367 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:06.450367 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:06.450367 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:06.450367 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 255/516] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:06.450367 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41c821-9dbd-473f-97d5-efb56c410e7c None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:06.478358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e301b73-8499-4704-9264-ca8950abf0b0 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:06.511519 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab15e67-0911-4bac-aeb0-5bbf49d13e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:06.512130 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab15e67-0911-4bac-aeb0-5bbf49d13e3e None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:06.512494 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab15e67-0911-4bac-aeb0-5bbf49d13e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:06.512746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab15e67-0911-4bac-aeb0-5bbf49d13e3e None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:06.513390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab15e67-0911-4bac-aeb0-5bbf49d13e3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:06.513690 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab15e67-0911-4bac-aeb0-5bbf49d13e3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:06.515102 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab15e67-0911-4bac-aeb0-5bbf49d13e3e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:06.527647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 262/517] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:06.528436 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab15e67-0911-4bac-aeb0-5bbf49d13e3e None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:06.549836 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41c821-9dbd-473f-97d5-efb56c410e7c None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:06.583217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd208d1-b0ad-4edd-8034-a187c381a77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:06.583831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd208d1-b0ad-4edd-8034-a187c381a77b None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:06.584103 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd208d1-b0ad-4edd-8034-a187c381a77b None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:06.584371 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd208d1-b0ad-4edd-8034-a187c381a77b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:06.595203 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd208d1-b0ad-4edd-8034-a187c381a77b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:06.595949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd208d1-b0ad-4edd-8034-a187c381a77b None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:06.602292 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:06.610568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-7fd208d1-b0ad-4edd-8034-a187c381a77b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:06.610568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:06.610568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:06.610568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:06.610568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:06.610568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:06.610568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:06.610568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:06.610568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:06.610568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:06.610568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:06.610568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:06.610568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:06.610568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:06.610568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:06.610568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:06.611510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:06.611510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:06.611510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:06.611510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:06.611510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:06.611510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:06.611510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:06.611510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:06.611510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:06.611510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:06.611510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:06.611510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:06.611510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:06.611510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:06.611510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:06.611510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:06.611510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:06.612905 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:06.612905 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:06.612905 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:06.612905 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 256/518] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:06.614363 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd208d1-b0ad-4edd-8034-a187c381a77b None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:06.625550 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab15e67-0911-4bac-aeb0-5bbf49d13e3e None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:06.648105 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:06.660094 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0bc03-07d4-4d09-8aba-f495eec0ee17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:06.660927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0bc03-07d4-4d09-8aba-f495eec0ee17 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:06.661205 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0bc03-07d4-4d09-8aba-f495eec0ee17 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:06.661480 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0bc03-07d4-4d09-8aba-f495eec0ee17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:06.671217 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0bc03-07d4-4d09-8aba-f495eec0ee17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:06.672655 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0bc03-07d4-4d09-8aba-f495eec0ee17 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:06.687343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-03e0bc03-07d4-4d09-8aba-f495eec0ee17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:06.687343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:06.687343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:06.687343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:06.687343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:06.687343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:06.687343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:06.687343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:06.687343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:06.687343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:06.687343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:06.687343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:06.687343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:06.687343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:06.687343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:06.687343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:06.688373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:06.688373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:06.688373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:06.688373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:06.688373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:06.688373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:06.688373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:06.688373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:06.688373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:06.688373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:06.688373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:06.688373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:06.688373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:06.688373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:06.688373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:06.688373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:06.688373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:06.693863 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:06.693863 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:06.693863 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:06.693863 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 263/519] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:06.693863 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0bc03-07d4-4d09-8aba-f495eec0ee17 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:06.711494 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd208d1-b0ad-4edd-8034-a187c381a77b None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:06.744989 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bdb02-a4b6-4af0-9591-338883d87dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:06.745584 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bdb02-a4b6-4af0-9591-338883d87dcb None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:06.745863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bdb02-a4b6-4af0-9591-338883d87dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:06.746160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bdb02-a4b6-4af0-9591-338883d87dcb None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:06.746782 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bdb02-a4b6-4af0-9591-338883d87dcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:06.747076 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bdb02-a4b6-4af0-9591-338883d87dcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:06.748458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bdb02-a4b6-4af0-9591-338883d87dcb None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:06.760088 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 257/520] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:06.760694 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bdb02-a4b6-4af0-9591-338883d87dcb None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:06.794265 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0bc03-07d4-4d09-8aba-f495eec0ee17 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:06.822382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:06.828565 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e3937-9ee1-44e6-8652-44731fd6293e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:06.829168 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e3937-9ee1-44e6-8652-44731fd6293e None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:06.829463 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e3937-9ee1-44e6-8652-44731fd6293e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:06.829722 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e3937-9ee1-44e6-8652-44731fd6293e None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:06.830385 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e3937-9ee1-44e6-8652-44731fd6293e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:06.830688 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e3937-9ee1-44e6-8652-44731fd6293e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:06.832095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e3937-9ee1-44e6-8652-44731fd6293e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:06.843576 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 264/521] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:06.844406 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e3937-9ee1-44e6-8652-44731fd6293e None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:06.853823 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:06.859938 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bdb02-a4b6-4af0-9591-338883d87dcb None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:06.894305 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e01e39-b237-4bbf-93d1-d3d930e2dafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:06.894901 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e01e39-b237-4bbf-93d1-d3d930e2dafb None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:06.895173 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e01e39-b237-4bbf-93d1-d3d930e2dafb None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:06.895435 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e01e39-b237-4bbf-93d1-d3d930e2dafb None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:06.896061 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e01e39-b237-4bbf-93d1-d3d930e2dafb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:06.897484 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e01e39-b237-4bbf-93d1-d3d930e2dafb None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:06.917423 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 258/522] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 14 01:54:06 2022] POST /identity/v3/users => generated 247 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:54:06.918240 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e01e39-b237-4bbf-93d1-d3d930e2dafb None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:06.959884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e3937-9ee1-44e6-8652-44731fd6293e None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:07.001788 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf6723-e568-4654-ba28-6540b0830e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:07.002518 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf6723-e568-4654-ba28-6540b0830e1a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:07.002945 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf6723-e568-4654-ba28-6540b0830e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:07.003219 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf6723-e568-4654-ba28-6540b0830e1a None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:07.003875 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf6723-e568-4654-ba28-6540b0830e1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:07.004216 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf6723-e568-4654-ba28-6540b0830e1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:07.005685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf6723-e568-4654-ba28-6540b0830e1a None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:07.018989 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 265/523] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:07.020038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf6723-e568-4654-ba28-6540b0830e1a None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:07.032204 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e01e39-b237-4bbf-93d1-d3d930e2dafb None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:07.069048 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719a6d7-7d1a-4f97-a201-69c7770656fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:07.069668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719a6d7-7d1a-4f97-a201-69c7770656fd None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:07.069963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719a6d7-7d1a-4f97-a201-69c7770656fd None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:07.070241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719a6d7-7d1a-4f97-a201-69c7770656fd None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:07.070889 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7719a6d7-7d1a-4f97-a201-69c7770656fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:07.071185 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7719a6d7-7d1a-4f97-a201-69c7770656fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:07.072593 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7719a6d7-7d1a-4f97-a201-69c7770656fd None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:07.076989 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:07.087506 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 259/524] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:07.088408 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719a6d7-7d1a-4f97-a201-69c7770656fd None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:07.104150 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:07.142666 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf6723-e568-4654-ba28-6540b0830e1a None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:07.178706 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0237a4-4693-4239-a752-4cd2cf9dffe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:07.179325 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0237a4-4693-4239-a752-4cd2cf9dffe8 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:07.179600 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0237a4-4693-4239-a752-4cd2cf9dffe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:07.179867 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0237a4-4693-4239-a752-4cd2cf9dffe8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:07.186816 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0237a4-4693-4239-a752-4cd2cf9dffe8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:07.188499 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0237a4-4693-4239-a752-4cd2cf9dffe8 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:07.197441 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719a6d7-7d1a-4f97-a201-69c7770656fd None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:07.202470 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-ce0237a4-4693-4239-a752-4cd2cf9dffe8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:07.202470 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:07.202470 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:07.202470 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:07.202470 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:07.202470 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:07.202470 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:07.202470 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:07.202470 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:07.202470 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:07.202470 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:07.202470 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:07.202470 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:07.202470 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:07.202470 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:07.203505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:07.203505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:07.203505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:07.203505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:07.203505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:07.203505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:07.203505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:07.203505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:07.203505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:07.203505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:07.203505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:07.203505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:07.203505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:07.203505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:07.203505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:07.203505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:07.203505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:07.210270 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:07.210270 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:07.210270 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:07.210270 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:07.210270 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:07.210270 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:07.210270 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:07.210270 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:07.210270 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 266/525] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:07.210270 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0237a4-4693-4239-a752-4cd2cf9dffe8 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:07.234487 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6fcc7-5684-47fe-a544-906a26dc14f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:07.235284 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6fcc7-5684-47fe-a544-906a26dc14f5 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:07.235631 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6fcc7-5684-47fe-a544-906a26dc14f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:07.235970 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6fcc7-5684-47fe-a544-906a26dc14f5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:07.243302 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6fcc7-5684-47fe-a544-906a26dc14f5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:07.244923 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6fcc7-5684-47fe-a544-906a26dc14f5 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:07.259411 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-95d6fcc7-5684-47fe-a544-906a26dc14f5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 01:54:07.259411 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:07.259411 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:07.259411 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:07.259411 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:07.259411 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:07.259411 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:07.259411 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:07.259411 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:07.259411 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:07.259411 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:07.259411 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:07.259411 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:07.259411 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:07.259411 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:07.259411 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:07.260358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:07.260358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:07.260358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:07.260358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:07.260358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:07.260358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:07.260358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:07.260358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:07.260358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:07.260358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:07.260358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:07.260358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:07.260358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:07.260358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:07.260358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:07.260358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:07.260358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:07.260358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:07.261270 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:07.261270 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:07.261270 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:07.261270 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:07.261270 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:07.261270 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:07.261270 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:07.261270 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:07.261270 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 01:54:07.261270 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:07.261270 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 260/526] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:54:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:07.261270 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6fcc7-5684-47fe-a544-906a26dc14f5 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:07.329431 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0237a4-4693-4239-a752-4cd2cf9dffe8 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:07.341289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:07.344244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:07.373917 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f452c-6b19-4171-94d5-785be0cfe88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:07.374576 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f452c-6b19-4171-94d5-785be0cfe88a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:07.374787 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f452c-6b19-4171-94d5-785be0cfe88a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:07.375060 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f452c-6b19-4171-94d5-785be0cfe88a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:07.379436 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6fcc7-5684-47fe-a544-906a26dc14f5 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:07.385408 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f452c-6b19-4171-94d5-785be0cfe88a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:07.386619 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f452c-6b19-4171-94d5-785be0cfe88a None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:07.400150 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-fa3f452c-6b19-4171-94d5-785be0cfe88a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:07.400150 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:07.400150 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:07.400150 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:07.400150 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:07.400150 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:07.400150 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:07.400150 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:07.400150 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:07.400150 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:07.400150 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:07.400150 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:07.400150 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:07.400150 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:07.400150 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:07.401232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:07.401232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:07.401232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:07.401232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:07.401232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:07.401232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:07.401232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:07.401232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:07.401232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:07.401232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:07.401232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:07.401232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:07.401232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:07.401232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:07.401232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:07.401232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:07.401232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:07.402339 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:07.402339 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:07.402339 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:07.402339 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:07.402339 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:07.402339 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:07.402339 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:07.402339 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:07.408737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 267/527] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:07.408737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f452c-6b19-4171-94d5-785be0cfe88a None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:07.428749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e666a-0607-4897-8849-dbb15f9d4c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:07.429515 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e666a-0607-4897-8849-dbb15f9d4c02 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:07.430049 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e666a-0607-4897-8849-dbb15f9d4c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:07.430585 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e666a-0607-4897-8849-dbb15f9d4c02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:07.439268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e666a-0607-4897-8849-dbb15f9d4c02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:07.440754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e666a-0607-4897-8849-dbb15f9d4c02 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:07.453045 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-7f4e666a-0607-4897-8849-dbb15f9d4c02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:07.453045 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:07.453045 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:07.453045 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:07.453045 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:07.453045 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:07.453045 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:07.453045 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:07.453045 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:07.453045 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:07.453045 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:07.453045 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:07.453045 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:07.453045 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:07.453045 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:07.453935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:07.453935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:07.453935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:07.453935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:07.453935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:07.453935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:07.453935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:07.453935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:07.453935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:07.453935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:07.453935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:07.453935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:07.453935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:07.453935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:07.453935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:07.453935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:07.453935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:07.454881 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:07.454881 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:07.454881 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:07.454881 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:07.454881 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:07.454881 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:07.454881 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:07.454881 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:07.459769 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 261/528] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:07.460299 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e666a-0607-4897-8849-dbb15f9d4c02 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:07.516565 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f452c-6b19-4171-94d5-785be0cfe88a None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:07.553289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd916b-dda0-4b16-982d-0d18531bd900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:07.553901 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd916b-dda0-4b16-982d-0d18531bd900 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:07.554210 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd916b-dda0-4b16-982d-0d18531bd900 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:07.554483 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd916b-dda0-4b16-982d-0d18531bd900 None admin] PATH_INFO: `/v3/projects` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:07.555163 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd916b-dda0-4b16-982d-0d18531bd900 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:07.555466 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd916b-dda0-4b16-982d-0d18531bd900 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:07.556904 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd916b-dda0-4b16-982d-0d18531bd900 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:07.562734 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e666a-0607-4897-8849-dbb15f9d4c02 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:07.574374 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 268/529] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:07.575018 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd916b-dda0-4b16-982d-0d18531bd900 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:07.600166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:07.607414 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:07.607689 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3845a62-adb7-409d-bc80-11ba2287fdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:07.608324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3845a62-adb7-409d-bc80-11ba2287fdb4 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:07.608561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3845a62-adb7-409d-bc80-11ba2287fdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:07.608828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3845a62-adb7-409d-bc80-11ba2287fdb4 None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:07.609555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3845a62-adb7-409d-bc80-11ba2287fdb4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:07.609920 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3845a62-adb7-409d-bc80-11ba2287fdb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:07.611371 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3845a62-adb7-409d-bc80-11ba2287fdb4 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:07.654068 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 262/530] 10.222.0.15 () {56 vars in 1139 bytes} [Wed Sep 14 01:54:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:07.654068 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3845a62-adb7-409d-bc80-11ba2287fdb4 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:07.686065 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd916b-dda0-4b16-982d-0d18531bd900 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:07.725061 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7add66-15d7-41a5-8bfe-4a16ca54f83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:07.725664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7add66-15d7-41a5-8bfe-4a16ca54f83b None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:07.725936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7add66-15d7-41a5-8bfe-4a16ca54f83b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:07.726249 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7add66-15d7-41a5-8bfe-4a16ca54f83b None admin] PATH_INFO: `/v3/projects` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:07.728069 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7add66-15d7-41a5-8bfe-4a16ca54f83b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:07.728069 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7add66-15d7-41a5-8bfe-4a16ca54f83b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:07.728608 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7add66-15d7-41a5-8bfe-4a16ca54f83b None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:07.741361 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 269/531] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:07.742201 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7add66-15d7-41a5-8bfe-4a16ca54f83b None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:07.769290 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3845a62-adb7-409d-bc80-11ba2287fdb4 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:07.805780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6807433-8646-4f9c-bbfc-94888427b477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:07.806493 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6807433-8646-4f9c-bbfc-94888427b477 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:07.806778 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6807433-8646-4f9c-bbfc-94888427b477 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:07.807054 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6807433-8646-4f9c-bbfc-94888427b477 None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:07.807737 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6807433-8646-4f9c-bbfc-94888427b477 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:07.808041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6807433-8646-4f9c-bbfc-94888427b477 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:07.809479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6807433-8646-4f9c-bbfc-94888427b477 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:07.833798 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 263/532] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:07.834598 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6807433-8646-4f9c-bbfc-94888427b477 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:07.837870 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:07.850022 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7add66-15d7-41a5-8bfe-4a16ca54f83b None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:07.886119 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d72a4-cd7b-481e-a41d-9322c8216599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:07.886730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d72a4-cd7b-481e-a41d-9322c8216599 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:07.887005 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d72a4-cd7b-481e-a41d-9322c8216599 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:07.887273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d72a4-cd7b-481e-a41d-9322c8216599 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:07.887988 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d72a4-cd7b-481e-a41d-9322c8216599 None admin] RBAC: Adding query filter params (role.id=e962a2a280224f9fbcd0fdc02262993d, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=f721a3a4eca941989685448a87bb9a19) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:07.888294 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d72a4-cd7b-481e-a41d-9322c8216599 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:07.889160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:07.889762 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d72a4-cd7b-481e-a41d-9322c8216599 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:07.906108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 270/533] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:54:07 2022] GET /identity/v3/role_assignments?user.id=f721a3a4eca941989685448a87bb9a19&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=e962a2a280224f9fbcd0fdc02262993d => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:07.906108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d72a4-cd7b-481e-a41d-9322c8216599 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:07.958501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6807433-8646-4f9c-bbfc-94888427b477 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:07.997251 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32eabda-9003-4b53-bcc9-a829881898c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:07.998676 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32eabda-9003-4b53-bcc9-a829881898c8 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:07.999136 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32eabda-9003-4b53-bcc9-a829881898c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:07.999670 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32eabda-9003-4b53-bcc9-a829881898c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:08.036088 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d72a4-cd7b-481e-a41d-9322c8216599 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:08.075649 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373ab94-8c32-4e27-b039-02d4c429d5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:08.076266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373ab94-8c32-4e27-b039-02d4c429d5a9 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:08.076543 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373ab94-8c32-4e27-b039-02d4c429d5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:08.076814 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373ab94-8c32-4e27-b039-02d4c429d5a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:08.077558 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373ab94-8c32-4e27-b039-02d4c429d5a9 None admin] RBAC: Adding query filter params (role.id=e962a2a280224f9fbcd0fdc02262993d, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=5a5191afa5f246269d5b4a60972332a6) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:08.077868 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373ab94-8c32-4e27-b039-02d4c429d5a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:08.079560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373ab94-8c32-4e27-b039-02d4c429d5a9 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:08.092212 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 271/534] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:54:07 2022] GET /identity/v3/role_assignments?user.id=5a5191afa5f246269d5b4a60972332a6&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=e962a2a280224f9fbcd0fdc02262993d => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:08.093329 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373ab94-8c32-4e27-b039-02d4c429d5a9 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:08.110676 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32eabda-9003-4b53-bcc9-a829881898c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:08.113645 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32eabda-9003-4b53-bcc9-a829881898c8 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:08.133165 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:08.182954 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:08.236873 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 264/535] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:54:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:54:08.244463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32eabda-9003-4b53-bcc9-a829881898c8 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:08.262633 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373ab94-8c32-4e27-b039-02d4c429d5a9 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:08.337010 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076414dc-0787-4f4e-9364-b5b9b1c4af0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:08.338482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076414dc-0787-4f4e-9364-b5b9b1c4af0f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:08.338965 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076414dc-0787-4f4e-9364-b5b9b1c4af0f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:08.343033 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076414dc-0787-4f4e-9364-b5b9b1c4af0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:08.344158 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076414dc-0787-4f4e-9364-b5b9b1c4af0f None admin] RBAC: Adding query filter params (role.id=e962a2a280224f9fbcd0fdc02262993d, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=c6d70ad7054e40c48f8751a3d4fed787) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:08.344722 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076414dc-0787-4f4e-9364-b5b9b1c4af0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:08.346464 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076414dc-0787-4f4e-9364-b5b9b1c4af0f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:08.360480 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 272/536] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:54:08 2022] GET /identity/v3/role_assignments?user.id=c6d70ad7054e40c48f8751a3d4fed787&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=e962a2a280224f9fbcd0fdc02262993d => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:08.419177 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32eabda-9003-4b53-bcc9-a829881898c8 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:08.468950 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c11f0-cb6d-449a-9815-20593bc49418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:08.469959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c11f0-cb6d-449a-9815-20593bc49418 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:08.470544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c11f0-cb6d-449a-9815-20593bc49418 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:08.471029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c11f0-cb6d-449a-9815-20593bc49418 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:08.490307 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c11f0-cb6d-449a-9815-20593bc49418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:08.492148 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c11f0-cb6d-449a-9815-20593bc49418 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:08.512223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-971c11f0-cb6d-449a-9815-20593bc49418 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:08.512223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:08.512223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:08.512223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:08.512223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:08.512223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:08.512223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:08.512223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:08.512223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:08.512223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:08.512223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:08.512223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:08.512223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:08.512223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:08.512223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:08.512223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:08.513202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:08.513202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:08.513202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:08.513202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:08.513202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:08.513202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:08.513202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:08.513202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:08.513202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:08.513202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:08.513202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:08.513202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:08.513202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:08.513202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:08.513202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:08.513202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:08.513202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:08.517218 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:08.517218 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:08.517218 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:08.519145 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 265/537] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:08.524460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076414dc-0787-4f4e-9364-b5b9b1c4af0f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:08.632466 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:08.661797 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076414dc-0787-4f4e-9364-b5b9b1c4af0f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:08.711621 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b8445-422e-4b0a-967c-2a569ae15dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:08.712251 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b8445-422e-4b0a-967c-2a569ae15dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:08.712528 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b8445-422e-4b0a-967c-2a569ae15dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:08.712795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b8445-422e-4b0a-967c-2a569ae15dd3 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:08.713430 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b8445-422e-4b0a-967c-2a569ae15dd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:08.713725 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b8445-422e-4b0a-967c-2a569ae15dd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:08.715125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b8445-422e-4b0a-967c-2a569ae15dd3 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:08.731251 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 273/538] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:08.736461 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c11f0-cb6d-449a-9815-20593bc49418 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:08.758389 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:08.845934 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c11f0-cb6d-449a-9815-20593bc49418 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:08.889024 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e0afc3-d723-48a5-8bc8-6457f2b8ee6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:08.889661 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0afc3-d723-48a5-8bc8-6457f2b8ee6d None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:08.889935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0afc3-d723-48a5-8bc8-6457f2b8ee6d None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:08.890231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0afc3-d723-48a5-8bc8-6457f2b8ee6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:08.901281 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e0afc3-d723-48a5-8bc8-6457f2b8ee6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:08.902765 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e0afc3-d723-48a5-8bc8-6457f2b8ee6d None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:08.915807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-48e0afc3-d723-48a5-8bc8-6457f2b8ee6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:08.915807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:08.915807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:08.915807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:08.915807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:08.915807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:08.915807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:08.915807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:08.915807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:08.915807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:08.915807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:08.915807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:08.915807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:08.915807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:08.915807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:08.916726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:08.916726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:08.916726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:08.916726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:08.916726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:08.916726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:08.916726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:08.916726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:08.916726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:08.916726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:08.916726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:08.916726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:08.916726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:08.916726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:08.916726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:08.916726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:08.916726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:08.917607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:08.917607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:08.917607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:08.917607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:08.917607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:08.917607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:08.917607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:08.917607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:08.922620 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b8445-422e-4b0a-967c-2a569ae15dd3 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:08.934968 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 266/539] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:09.064006 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b8445-422e-4b0a-967c-2a569ae15dd3 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:09.115769 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c461fb06-6a60-43f2-966d-a945dd78b5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:09.116389 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461fb06-6a60-43f2-966d-a945dd78b5fb None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:09.116661 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461fb06-6a60-43f2-966d-a945dd78b5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:09.116928 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461fb06-6a60-43f2-966d-a945dd78b5fb None admin] PATH_INFO: `/v3/projects` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:09.117615 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c461fb06-6a60-43f2-966d-a945dd78b5fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:09.117915 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c461fb06-6a60-43f2-966d-a945dd78b5fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:09.119381 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c461fb06-6a60-43f2-966d-a945dd78b5fb None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:09.132197 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 274/540] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:09.139292 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e0afc3-d723-48a5-8bc8-6457f2b8ee6d None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:09.147628 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d1bc4-7a2c-4e5e-9978-446e763ea64a None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:09.147914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d1bc4-7a2c-4e5e-9978-446e763ea64a None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:09.148191 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d1bc4-7a2c-4e5e-9978-446e763ea64a None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:09.149507 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 275/541] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:54:09.154565 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a45cd-a9b8-432f-9551-0c5b0d54c217 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:09.154849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a45cd-a9b8-432f-9551-0c5b0d54c217 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:09.155125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a45cd-a9b8-432f-9551-0c5b0d54c217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:09.170004 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:09.176310 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-e63a45cd-a9b8-432f-9551-0c5b0d54c217 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:09.177832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-e63a45cd-a9b8-432f-9551-0c5b0d54c217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:09.249526 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 276/542] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:54:09.312447 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:09.365444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e0afc3-d723-48a5-8bc8-6457f2b8ee6d None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:09.406104 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb8438-92e2-4077-8a0e-d805d7f11d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:09.406712 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb8438-92e2-4077-8a0e-d805d7f11d20 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:09.406989 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb8438-92e2-4077-8a0e-d805d7f11d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:09.407258 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb8438-92e2-4077-8a0e-d805d7f11d20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:09.407983 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb8438-92e2-4077-8a0e-d805d7f11d20 None admin] RBAC: Adding query filter params (role.id=e962a2a280224f9fbcd0fdc02262993d, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=bf86028f85af444380293d2387928eda) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:09.408284 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb8438-92e2-4077-8a0e-d805d7f11d20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:09.410053 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb8438-92e2-4077-8a0e-d805d7f11d20 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:09.416135 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 267/543] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:54:09 2022] GET /identity/v3/role_assignments?user.id=bf86028f85af444380293d2387928eda&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=e962a2a280224f9fbcd0fdc02262993d => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:09.597606 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a45cd-a9b8-432f-9551-0c5b0d54c217 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:09.712247 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a45cd-a9b8-432f-9551-0c5b0d54c217 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:09.746375 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:09.750091 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9fdcbf-f693-458d-b947-5bd81fe8fa97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:09.751227 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fdcbf-f693-458d-b947-5bd81fe8fa97 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:09.751889 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fdcbf-f693-458d-b947-5bd81fe8fa97 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:09.752560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fdcbf-f693-458d-b947-5bd81fe8fa97 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:09.764431 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9fdcbf-f693-458d-b947-5bd81fe8fa97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:09.769921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-6b9fdcbf-f693-458d-b947-5bd81fe8fa97 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:09.769921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:09.769921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:09.769921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:09.769921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:09.769921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:09.769921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:09.769921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:09.769921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:09.769921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:09.769921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:09.769921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:09.769921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:09.769921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:09.769921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:09.769921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:09.770888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:09.770888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:09.770888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:09.770888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:09.770888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:09.770888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:09.770888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:09.770888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:09.770888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:09.770888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:09.770888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:09.770888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:09.770888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:09.770888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:09.770888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:09.770888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:09.771954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:09.771954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:09.771954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:09.771954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:09.771954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:09.771954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:09.771954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:09.771954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:09.771954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:09.771954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:09.771954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:09.771954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:09.771954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 277/544] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:54:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:09.778684 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb8438-92e2-4077-8a0e-d805d7f11d20 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:09.886451 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb8438-92e2-4077-8a0e-d805d7f11d20 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:09.913026 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:09.922197 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909e48b-142b-44c8-a1d5-0d3de29cb4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:09.922806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909e48b-142b-44c8-a1d5-0d3de29cb4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:09.923086 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909e48b-142b-44c8-a1d5-0d3de29cb4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:09.923361 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909e48b-142b-44c8-a1d5-0d3de29cb4a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:09.924020 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909e48b-142b-44c8-a1d5-0d3de29cb4a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:09.924317 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909e48b-142b-44c8-a1d5-0d3de29cb4a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:09.926226 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909e48b-142b-44c8-a1d5-0d3de29cb4a2 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:09.934443 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 268/545] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 14 01:54:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:09.940184 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9fdcbf-f693-458d-b947-5bd81fe8fa97 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:10.042851 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9fdcbf-f693-458d-b947-5bd81fe8fa97 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:10.077254 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4cf7e-f2ae-4220-8eea-f3760220b283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:10.077880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4cf7e-f2ae-4220-8eea-f3760220b283 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:10.078187 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4cf7e-f2ae-4220-8eea-f3760220b283 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:10.078456 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4cf7e-f2ae-4220-8eea-f3760220b283 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:10.088680 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4cf7e-f2ae-4220-8eea-f3760220b283 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:10.090214 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4cf7e-f2ae-4220-8eea-f3760220b283 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:10.105390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-f0a4cf7e-f2ae-4220-8eea-f3760220b283 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:10.105390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:10.105390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:10.105390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:10.105390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:10.105390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:10.105390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:10.105390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:10.105390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:10.105390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:10.105390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:10.105390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:10.105390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:10.105390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:10.105390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:10.105390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:10.106367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:10.106367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:10.106367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:10.106367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:10.106367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:10.106367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:10.106367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:10.106367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:10.106367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:10.106367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:10.106367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:10.106367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:10.106367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:10.106367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:10.106367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:10.106367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:10.106367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:10.107835 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:10.107835 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:10.107835 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:10.107835 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 278/546] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:10.112470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909e48b-142b-44c8-a1d5-0d3de29cb4a2 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:10.214473 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909e48b-142b-44c8-a1d5-0d3de29cb4a2 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:10.253700 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e5a6d-433c-43e1-869d-d17b33cb159b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:10.254347 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e5a6d-433c-43e1-869d-d17b33cb159b None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:10.254626 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e5a6d-433c-43e1-869d-d17b33cb159b None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:10.254894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e5a6d-433c-43e1-869d-d17b33cb159b None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:10.255538 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e5a6d-433c-43e1-869d-d17b33cb159b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:10.255838 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e5a6d-433c-43e1-869d-d17b33cb159b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:10.257465 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e5a6d-433c-43e1-869d-d17b33cb159b None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:10.275764 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 269/547] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:10.281357 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4cf7e-f2ae-4220-8eea-f3760220b283 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:10.330656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:10.392151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4cf7e-f2ae-4220-8eea-f3760220b283 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:10.400047 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05650c53-1333-4b06-b381-fb3b26eaf80a None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:10.400599 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05650c53-1333-4b06-b381-fb3b26eaf80a None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:10.401091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05650c53-1333-4b06-b381-fb3b26eaf80a None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:10.402652 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 270/548] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:54:10.407733 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61efbe3-04b6-42fc-9f6a-8a7bbc96aae7 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:10.408014 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61efbe3-04b6-42fc-9f6a-8a7bbc96aae7 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:10.408282 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61efbe3-04b6-42fc-9f6a-8a7bbc96aae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:10.436251 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806ba99-f045-4c29-b42a-27b8179aa2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:10.437247 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806ba99-f045-4c29-b42a-27b8179aa2ca None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:10.438146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806ba99-f045-4c29-b42a-27b8179aa2ca None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:10.438661 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806ba99-f045-4c29-b42a-27b8179aa2ca None admin] PATH_INFO: `/v3/users/swift` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:10.454764 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-d61efbe3-04b6-42fc-9f6a-8a7bbc96aae7 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:10.456792 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-d61efbe3-04b6-42fc-9f6a-8a7bbc96aae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:10.457929 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e806ba99-f045-4c29-b42a-27b8179aa2ca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:10.459622 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e806ba99-f045-4c29-b42a-27b8179aa2ca None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:10.483013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-e806ba99-f045-4c29-b42a-27b8179aa2ca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:54:10.483013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:10.483013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:10.483013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:10.483013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:10.483013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:10.483013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:10.483013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:10.483013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:10.483013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:10.483013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:10.483013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:10.483013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:10.483013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:10.483013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:10.483013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:10.484521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:10.484521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:10.484521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:10.484521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:10.484521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:10.484521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:10.484521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:10.484521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:10.484521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:10.484521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:10.484521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:10.484521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:10.484521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:10.484521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:10.484521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:10.484521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:10.484521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:10.484521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:10.485623 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:10.485623 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:10.485623 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:10.485623 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:10.485623 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:10.485623 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:10.485623 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:10.485623 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:10.485623 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:54:10.485623 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:10.493135 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 279/549] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:54:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:10.500739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806ba99-f045-4c29-b42a-27b8179aa2ca None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:10.563543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:10.615886 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806ba99-f045-4c29-b42a-27b8179aa2ca None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:10.618843 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:10.630321 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 271/550] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:54:10.666722 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d40fe0-0757-4b8f-bf9e-9b9020d26dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:10.667339 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d40fe0-0757-4b8f-bf9e-9b9020d26dec None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:10.667617 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d40fe0-0757-4b8f-bf9e-9b9020d26dec None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:10.667886 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d40fe0-0757-4b8f-bf9e-9b9020d26dec None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:10.668679 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d40fe0-0757-4b8f-bf9e-9b9020d26dec None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:10.668986 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d40fe0-0757-4b8f-bf9e-9b9020d26dec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:10.671236 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d40fe0-0757-4b8f-bf9e-9b9020d26dec None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:10.690795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 280/551] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Sep 14 01:54:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:10.696429 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61efbe3-04b6-42fc-9f6a-8a7bbc96aae7 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:10.850186 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61efbe3-04b6-42fc-9f6a-8a7bbc96aae7 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:10.911860 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84004d82-d73d-441d-bd09-764b4adffa67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:10.912414 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84004d82-d73d-441d-bd09-764b4adffa67 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:10.912691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84004d82-d73d-441d-bd09-764b4adffa67 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:10.912961 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84004d82-d73d-441d-bd09-764b4adffa67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:10.970584 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:10.981448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9366b6-6578-4111-833e-ae6d89f486de None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:10.981731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9366b6-6578-4111-833e-ae6d89f486de None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:10.982023 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9366b6-6578-4111-833e-ae6d89f486de None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:10.983869 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 281/552] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:54:10.988882 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e123b24-ab92-4f24-b638-db22f18f5a0e None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:10.989167 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e123b24-ab92-4f24-b638-db22f18f5a0e None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:10.989436 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e123b24-ab92-4f24-b638-db22f18f5a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:10.991111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84004d82-d73d-441d-bd09-764b4adffa67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:10.992573 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84004d82-d73d-441d-bd09-764b4adffa67 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:11.025701 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-0e123b24-ab92-4f24-b638-db22f18f5a0e None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:11.026911 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-0e123b24-ab92-4f24-b638-db22f18f5a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:11.097341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 272/553] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:54:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:54:11.098274 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84004d82-d73d-441d-bd09-764b4adffa67 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:11.130092 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 282/554] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:54:11.131621 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e123b24-ab92-4f24-b638-db22f18f5a0e None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:11.181771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:11.204061 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84004d82-d73d-441d-bd09-764b4adffa67 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:11.242336 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f41d58-303e-4000-acd9-0f194cc4458d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:11.243301 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f41d58-303e-4000-acd9-0f194cc4458d None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:11.243729 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f41d58-303e-4000-acd9-0f194cc4458d None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:11.244149 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f41d58-303e-4000-acd9-0f194cc4458d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:11.277632 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f41d58-303e-4000-acd9-0f194cc4458d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:11.290831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-d8f41d58-303e-4000-acd9-0f194cc4458d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:11.290831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:11.290831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:11.290831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:11.290831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:11.290831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:11.290831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:11.290831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:11.290831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:11.290831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:11.290831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:11.290831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:11.290831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:11.290831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:11.290831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:11.290831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:11.291972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:11.291972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:11.291972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:11.291972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:11.291972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:11.291972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:11.291972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:11.291972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:11.291972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:11.291972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:11.291972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:11.291972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:11.291972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:11.291972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:11.291972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:11.291972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:11.293012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:11.293012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:11.293012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:11.293012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:11.293012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:11.293012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:11.293012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:11.293012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:11.293012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:11.293012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:11.293012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:11.293012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:11.295285 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 273/555] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:54:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:11.301713 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f41d58-303e-4000-acd9-0f194cc4458d None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:11.312556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:11.326560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e123b24-ab92-4f24-b638-db22f18f5a0e None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:11.371944 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d52cf3-1304-4616-a2ea-7dd917269d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:11.372592 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d52cf3-1304-4616-a2ea-7dd917269d57 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:11.372871 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d52cf3-1304-4616-a2ea-7dd917269d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:11.373143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d52cf3-1304-4616-a2ea-7dd917269d57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:11.386880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d52cf3-1304-4616-a2ea-7dd917269d57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:11.388435 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d52cf3-1304-4616-a2ea-7dd917269d57 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:11.415404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-42d52cf3-1304-4616-a2ea-7dd917269d57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:11.415404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:11.415404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:11.415404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:11.415404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:11.415404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:11.415404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:11.415404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:11.415404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:11.415404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:11.415404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:11.415404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:11.415404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:11.415404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:11.415404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:11.415404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:11.416446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:11.416446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:11.416446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:11.416446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:11.416446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:11.416446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:11.416446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:11.416446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:11.416446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:11.416446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:11.416446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:11.416446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:11.416446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:11.416446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:11.416446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:11.416446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:11.416446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:11.417344 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:11.417344 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:11.417344 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:11.417526 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 283/556] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:11.418789 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c63592-535c-4dab-8d69-49ed180d8a6e None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:11.419077 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c63592-535c-4dab-8d69-49ed180d8a6e None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:11.419349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c63592-535c-4dab-8d69-49ed180d8a6e None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:11.425263 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 284/557] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:11.425263 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c63592-535c-4dab-8d69-49ed180d8a6e None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:11.470783 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f41d58-303e-4000-acd9-0f194cc4458d None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:11.506724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4044cb1-cfea-434f-be34-0b69517d6fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:11.507332 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4044cb1-cfea-434f-be34-0b69517d6fbf None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:11.507603 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4044cb1-cfea-434f-be34-0b69517d6fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:11.507880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4044cb1-cfea-434f-be34-0b69517d6fbf None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:11.508529 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4044cb1-cfea-434f-be34-0b69517d6fbf None admin] RBAC: Adding query filter params (name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:11.508829 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4044cb1-cfea-434f-be34-0b69517d6fbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:11.510226 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4044cb1-cfea-434f-be34-0b69517d6fbf None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:11.516073 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:11.522238 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 274/558] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 14 01:54:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:11.523890 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3520b79-4a1c-4798-8e73-a00af99410e8 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:11.524248 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3520b79-4a1c-4798-8e73-a00af99410e8 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:11.524766 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3520b79-4a1c-4798-8e73-a00af99410e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:11.544837 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c63592-535c-4dab-8d69-49ed180d8a6e None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:11.546850 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-e3520b79-4a1c-4798-8e73-a00af99410e8 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:11.548038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-e3520b79-4a1c-4798-8e73-a00af99410e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:11.579997 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699f695-3a07-450c-b929-3cd974f412f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:11.581069 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699f695-3a07-450c-b929-3cd974f412f6 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:11.581597 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699f695-3a07-450c-b929-3cd974f412f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:11.581597 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699f695-3a07-450c-b929-3cd974f412f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:11.582340 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699f695-3a07-450c-b929-3cd974f412f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:11.582636 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699f695-3a07-450c-b929-3cd974f412f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:11.584006 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699f695-3a07-450c-b929-3cd974f412f6 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:11.600790 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 285/559] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:54:11.600790 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699f695-3a07-450c-b929-3cd974f412f6 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:11.601230 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:11.626737 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 275/560] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:54:11.628175 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3520b79-4a1c-4798-8e73-a00af99410e8 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:11.727761 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699f695-3a07-450c-b929-3cd974f412f6 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:11.728286 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3520b79-4a1c-4798-8e73-a00af99410e8 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:11.758542 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:11.766245 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceefb77-4bc9-470e-b7cb-c6ec9ad5d6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:11.766955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceefb77-4bc9-470e-b7cb-c6ec9ad5d6fa None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:11.767326 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceefb77-4bc9-470e-b7cb-c6ec9ad5d6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:11.767667 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceefb77-4bc9-470e-b7cb-c6ec9ad5d6fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:11.772895 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0611d27-fd65-4c27-b52a-61e0de954352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:11.773748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0611d27-fd65-4c27-b52a-61e0de954352 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:11.774250 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0611d27-fd65-4c27-b52a-61e0de954352 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:11.774705 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0611d27-fd65-4c27-b52a-61e0de954352 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:11.779304 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceefb77-4bc9-470e-b7cb-c6ec9ad5d6fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:11.780840 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceefb77-4bc9-470e-b7cb-c6ec9ad5d6fa None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:11.788452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0611d27-fd65-4c27-b52a-61e0de954352 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:11.794998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-c0611d27-fd65-4c27-b52a-61e0de954352 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:11.794998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:11.794998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:11.794998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:11.794998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:11.794998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:11.794998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:11.794998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:11.794998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:11.794998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:11.794998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:11.794998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:11.794998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:11.794998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:11.794998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:11.794998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:11.795970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:11.795970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:11.795970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:11.795970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:11.795970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:11.795970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:11.795970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:11.795970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:11.795970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:11.795970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:11.795970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:11.795970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:11.795970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:11.795970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:11.795970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:11.795970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:11.796855 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:11.796855 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:11.796855 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:11.796855 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:11.796855 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:11.796855 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:11.796855 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:11.796855 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:11.796855 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:11.796855 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:11.796855 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:11.796855 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:11.797594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-1ceefb77-4bc9-470e-b7cb-c6ec9ad5d6fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:11.797594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:11.797594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:11.797594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:11.797594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:11.797594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:11.797594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:11.797594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:11.797594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:11.797594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:11.797594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:11.797594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:11.797594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:11.797594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:11.797594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:11.797594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:11.798498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:11.798498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:11.798498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:11.798498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:11.798498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:11.798498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:11.798498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:11.798498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:11.798498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:11.798498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:11.798498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:11.798498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:11.798498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:11.798498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:11.798498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:11.798498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:11.798498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:11.800360 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:11.800360 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:11.800360 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:11.800360 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 276/561] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:11.801346 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceefb77-4bc9-470e-b7cb-c6ec9ad5d6fa None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:11.802516 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 286/562] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:54:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:11.810971 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0611d27-fd65-4c27-b52a-61e0de954352 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:11.875606 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:11.915732 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0611d27-fd65-4c27-b52a-61e0de954352 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:11.934456 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceefb77-4bc9-470e-b7cb-c6ec9ad5d6fa None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:11.954604 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233c7c2-cff1-4cf1-b096-34d7d22edc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:11.955211 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233c7c2-cff1-4cf1-b096-34d7d22edc41 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:11.955489 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233c7c2-cff1-4cf1-b096-34d7d22edc41 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:11.956297 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233c7c2-cff1-4cf1-b096-34d7d22edc41 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:11.956961 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233c7c2-cff1-4cf1-b096-34d7d22edc41 None admin] RBAC: Adding query filter params (name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:11.957263 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233c7c2-cff1-4cf1-b096-34d7d22edc41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:11.958670 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233c7c2-cff1-4cf1-b096-34d7d22edc41 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:11.967062 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 287/563] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 14 01:54:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:11.972223 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233c7c2-cff1-4cf1-b096-34d7d22edc41 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:11.974438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b217a7e-e663-4a4c-b649-243600683237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:11.975070 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b217a7e-e663-4a4c-b649-243600683237 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:11.975341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b217a7e-e663-4a4c-b649-243600683237 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:11.976278 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b217a7e-e663-4a4c-b649-243600683237 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:11.982505 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b217a7e-e663-4a4c-b649-243600683237 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:11.983912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b217a7e-e663-4a4c-b649-243600683237 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:11.997027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-4b217a7e-e663-4a4c-b649-243600683237 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:11.997027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:11.997027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:11.997027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:11.997027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:11.997027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:11.997027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:11.997027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:11.997027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:11.997027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:11.997027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:11.997027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:11.997027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:11.997027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:11.997027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:11.997955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:11.997955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:11.997955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:11.997955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:11.997955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:11.997955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:11.997955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:11.997955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:11.997955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:11.997955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:11.997955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:11.997955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:11.997955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:11.997955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:11.997955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:11.997955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:11.997955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:11.998986 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:11.998986 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:11.998986 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:11.998986 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:11.998986 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:11.998986 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:11.998986 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:11.998986 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:11.998986 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 277/564] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:12.000059 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b217a7e-e663-4a4c-b649-243600683237 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:12.045999 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:12.082837 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233c7c2-cff1-4cf1-b096-34d7d22edc41 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:12.107151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:12.108354 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b217a7e-e663-4a4c-b649-243600683237 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:12.116914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530a2b5-1dd1-4ed7-83ac-8bea96c73bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:12.117527 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530a2b5-1dd1-4ed7-83ac-8bea96c73bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:12.117805 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530a2b5-1dd1-4ed7-83ac-8bea96c73bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:12.118137 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530a2b5-1dd1-4ed7-83ac-8bea96c73bf5 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:12.118749 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c530a2b5-1dd1-4ed7-83ac-8bea96c73bf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:12.119053 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c530a2b5-1dd1-4ed7-83ac-8bea96c73bf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:12.120428 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c530a2b5-1dd1-4ed7-83ac-8bea96c73bf5 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:12.133298 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 288/565] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:12.134462 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530a2b5-1dd1-4ed7-83ac-8bea96c73bf5 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:12.164603 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876c28c-0517-411c-b3d5-f8eb0a521e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:12.165262 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876c28c-0517-411c-b3d5-f8eb0a521e07 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:12.165538 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876c28c-0517-411c-b3d5-f8eb0a521e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:12.165807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876c28c-0517-411c-b3d5-f8eb0a521e07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:12.177275 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9876c28c-0517-411c-b3d5-f8eb0a521e07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:12.182749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-9876c28c-0517-411c-b3d5-f8eb0a521e07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:12.182749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:12.182749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:12.182749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:12.182749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:12.182749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:12.182749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:12.182749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:12.182749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:12.182749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:12.182749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:12.182749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:12.182749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:12.182749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:12.182749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:12.182749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:12.186213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:12.186213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:12.186213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:12.186213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:12.186213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:12.186213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:12.186213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:12.186213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:12.186213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:12.186213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:12.186213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:12.186213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:12.186213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:12.186213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:12.186213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:12.186213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:12.187160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:12.187160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:12.187160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:12.187160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:12.187160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:12.187160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:12.187160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:12.187160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:12.187160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:12.187160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:12.187160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:12.187160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:12.187160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 278/566] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:54:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:12.187160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876c28c-0517-411c-b3d5-f8eb0a521e07 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:12.248942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530a2b5-1dd1-4ed7-83ac-8bea96c73bf5 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:12.286538 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e440b0a-ac96-44b4-b31a-6eb5a24b44b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:12.287158 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e440b0a-ac96-44b4-b31a-6eb5a24b44b4 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:12.287432 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e440b0a-ac96-44b4-b31a-6eb5a24b44b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:12.287699 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e440b0a-ac96-44b4-b31a-6eb5a24b44b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:12.287973 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876c28c-0517-411c-b3d5-f8eb0a521e07 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:12.289597 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:12.297710 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e440b0a-ac96-44b4-b31a-6eb5a24b44b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:12.299196 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e440b0a-ac96-44b4-b31a-6eb5a24b44b4 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:12.313811 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-9e440b0a-ac96-44b4-b31a-6eb5a24b44b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:12.313811 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:12.313811 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:12.313811 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:12.313811 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:12.313811 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:12.313811 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:12.313811 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:12.313811 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:12.313811 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:12.313811 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:12.313811 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:12.313811 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:12.313811 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:12.313811 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:12.313811 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:12.314819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:12.314819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:12.314819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:12.314819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:12.314819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:12.314819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:12.314819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:12.314819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:12.314819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:12.314819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:12.314819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:12.314819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:12.314819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:12.314819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:12.314819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:12.314819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:12.314819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:12.315776 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:12.315776 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:12.315776 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:12.315776 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 289/567] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:12.315776 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e440b0a-ac96-44b4-b31a-6eb5a24b44b4 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:12.326356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e28db-ae7a-4d74-bb78-daf554b40430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:12.326980 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e28db-ae7a-4d74-bb78-daf554b40430 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:12.327250 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e28db-ae7a-4d74-bb78-daf554b40430 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:12.327510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e28db-ae7a-4d74-bb78-daf554b40430 None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:12.328189 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e28db-ae7a-4d74-bb78-daf554b40430 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:12.328485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e28db-ae7a-4d74-bb78-daf554b40430 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:12.329889 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e28db-ae7a-4d74-bb78-daf554b40430 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:12.341414 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 279/568] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:12.342206 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e28db-ae7a-4d74-bb78-daf554b40430 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:12.356808 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:12.416807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e440b0a-ac96-44b4-b31a-6eb5a24b44b4 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:12.443737 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e28db-ae7a-4d74-bb78-daf554b40430 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:12.449824 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843ab25-7560-4c1e-86f0-cd8a50a8796c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:12.450487 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843ab25-7560-4c1e-86f0-cd8a50a8796c None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:12.450772 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843ab25-7560-4c1e-86f0-cd8a50a8796c None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:12.451046 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843ab25-7560-4c1e-86f0-cd8a50a8796c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:12.458566 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7843ab25-7560-4c1e-86f0-cd8a50a8796c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:12.460046 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7843ab25-7560-4c1e-86f0-cd8a50a8796c None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:12.471342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-7843ab25-7560-4c1e-86f0-cd8a50a8796c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:54:12.471342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:12.471342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:12.471342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:12.471342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:12.471342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:12.471342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:12.471342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:12.471342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:12.471342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:12.471342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:12.471342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:12.471342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:12.471342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:12.471342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:12.471342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:12.472382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:12.472382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:12.472382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:12.472382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:12.472382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:12.472382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:12.472382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:12.472382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:12.472382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:12.472382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:12.472382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:12.472382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:12.472382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:12.472382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:12.472382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:12.472382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:12.472382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:12.472382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:12.473342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:12.473342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:12.473342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:12.473342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:12.473342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:12.473342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:12.473342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:12.473342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:12.473342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:54:12.473342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:12.473342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 290/569] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Sep 14 01:54:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:12.473949 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2115c4f-ab61-4488-be4a-c44ff6152b6d None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:12.473949 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2115c4f-ab61-4488-be4a-c44ff6152b6d None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:12.474126 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2115c4f-ab61-4488-be4a-c44ff6152b6d None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:12.475338 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 291/570] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:12.475914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2115c4f-ab61-4488-be4a-c44ff6152b6d None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:12.483752 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bfb086-889a-4d30-8104-2c822a7708ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:12.484354 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bfb086-889a-4d30-8104-2c822a7708ca None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:12.484625 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bfb086-889a-4d30-8104-2c822a7708ca None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:12.484888 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bfb086-889a-4d30-8104-2c822a7708ca None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:12.485540 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bfb086-889a-4d30-8104-2c822a7708ca None admin] RBAC: Adding query filter params (name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:12.485831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bfb086-889a-4d30-8104-2c822a7708ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:12.487275 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bfb086-889a-4d30-8104-2c822a7708ca None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:12.495272 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 280/571] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 14 01:54:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:12.496055 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bfb086-889a-4d30-8104-2c822a7708ca None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:12.524702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:12.574293 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:12.579546 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2115c4f-ab61-4488-be4a-c44ff6152b6d None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:12.597707 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bfb086-889a-4d30-8104-2c822a7708ca None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:12.612749 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe17920-fcfc-4ec9-b844-9441e887ffd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:12.613352 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe17920-fcfc-4ec9-b844-9441e887ffd9 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:12.613628 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe17920-fcfc-4ec9-b844-9441e887ffd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:12.613895 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe17920-fcfc-4ec9-b844-9441e887ffd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:12.614555 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe17920-fcfc-4ec9-b844-9441e887ffd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:12.614852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe17920-fcfc-4ec9-b844-9441e887ffd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:12.616211 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe17920-fcfc-4ec9-b844-9441e887ffd9 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:12.627386 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 292/572] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:12.628228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe17920-fcfc-4ec9-b844-9441e887ffd9 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:12.629555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd40efb-046a-444e-8fed-7d67df16ffd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:12.630263 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd40efb-046a-444e-8fed-7d67df16ffd7 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:12.630538 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd40efb-046a-444e-8fed-7d67df16ffd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:12.630802 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd40efb-046a-444e-8fed-7d67df16ffd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:12.631800 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd40efb-046a-444e-8fed-7d67df16ffd7 None admin] RBAC: Adding query filter params (role.id=e962a2a280224f9fbcd0fdc02262993d, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=21f481c1f60043e9b0e25ba49b30715f) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:12.632297 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd40efb-046a-444e-8fed-7d67df16ffd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:12.633725 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd40efb-046a-444e-8fed-7d67df16ffd7 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:12.650012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 281/573] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:54:12 2022] GET /identity/v3/role_assignments?user.id=21f481c1f60043e9b0e25ba49b30715f&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=e962a2a280224f9fbcd0fdc02262993d => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:12.650012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239830f-c885-4d26-917c-4c7c99151035 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:12.650012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239830f-c885-4d26-917c-4c7c99151035 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:12.650012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239830f-c885-4d26-917c-4c7c99151035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:12.660568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-1239830f-c885-4d26-917c-4c7c99151035 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:12.661794 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-1239830f-c885-4d26-917c-4c7c99151035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:12.741542 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe17920-fcfc-4ec9-b844-9441e887ffd9 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:12.744992 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 282/574] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:54:12.746047 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239830f-c885-4d26-917c-4c7c99151035 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:12.801237 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:12.803851 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a155e73-8e5c-41c2-b1dd-a91292d6e594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:12.818009 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a155e73-8e5c-41c2-b1dd-a91292d6e594 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:12.818009 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a155e73-8e5c-41c2-b1dd-a91292d6e594 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:12.818009 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a155e73-8e5c-41c2-b1dd-a91292d6e594 None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:12.818009 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a155e73-8e5c-41c2-b1dd-a91292d6e594 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:12.818009 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a155e73-8e5c-41c2-b1dd-a91292d6e594 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:12.818009 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a155e73-8e5c-41c2-b1dd-a91292d6e594 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:12.834226 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 293/575] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Sep 14 01:54:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:12.835255 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a155e73-8e5c-41c2-b1dd-a91292d6e594 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:12.874930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:12.973133 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239830f-c885-4d26-917c-4c7c99151035 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:13.006650 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a155e73-8e5c-41c2-b1dd-a91292d6e594 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:13.030861 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2c530-4a95-402d-b97e-6806a561b300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:13.032169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2c530-4a95-402d-b97e-6806a561b300 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:13.032967 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2c530-4a95-402d-b97e-6806a561b300 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:13.033248 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2c530-4a95-402d-b97e-6806a561b300 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:13.047382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159e80c-a132-40fa-96fb-ca1f46e9444f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:13.048119 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159e80c-a132-40fa-96fb-ca1f46e9444f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:13.048398 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159e80c-a132-40fa-96fb-ca1f46e9444f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:13.048515 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2c530-4a95-402d-b97e-6806a561b300 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:13.048670 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159e80c-a132-40fa-96fb-ca1f46e9444f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:13.049968 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2c530-4a95-402d-b97e-6806a561b300 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:13.057630 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159e80c-a132-40fa-96fb-ca1f46e9444f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:13.059033 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159e80c-a132-40fa-96fb-ca1f46e9444f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:13.068546 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-63b2c530-4a95-402d-b97e-6806a561b300 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:13.068546 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:13.068546 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:13.068546 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:13.068546 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:13.068546 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:13.068546 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:13.068546 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:13.068546 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:13.068546 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:13.068546 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:13.068546 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:13.068546 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:13.068546 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:13.068546 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:13.068546 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:13.069532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:13.069532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:13.069532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:13.069532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:13.069532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:13.069532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:13.069532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:13.069532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:13.069532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:13.069532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:13.069532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:13.069532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:13.069532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:13.069532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:13.069532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:13.069532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:13.069532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:13.070493 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:13.070493 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:13.070493 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:13.071453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-b159e80c-a132-40fa-96fb-ca1f46e9444f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 14 01:54:13.071453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:13.071453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:13.071453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:13.071453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:13.071453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:13.071453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:13.071453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:13.071453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:13.071453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:13.071453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:13.071453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:13.071453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:13.071453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:13.071453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:13.071453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:13.072359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:13.072359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:13.072359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:13.072359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:13.072359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:13.072359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:13.072359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:13.072359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:13.072359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:13.072359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:13.072359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:13.072359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:13.072359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:13.072359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:13.072359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:13.072359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:13.072359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:13.072359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:13.074447 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:13.074447 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:13.074447 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:13.074447 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:13.074447 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:13.074447 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:13.074447 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:13.074447 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:13.074447 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 14 01:54:13.074447 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:13.074447 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 294/576] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:54:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:13.074447 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159e80c-a132-40fa-96fb-ca1f46e9444f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:13.076024 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 283/577] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:13.076024 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2c530-4a95-402d-b97e-6806a561b300 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:13.156766 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:13.171818 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:13.194742 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2c530-4a95-402d-b97e-6806a561b300 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:13.208611 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159e80c-a132-40fa-96fb-ca1f46e9444f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:13.231396 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cdc5b-29e6-48ff-baab-78f73505b7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:13.232022 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cdc5b-29e6-48ff-baab-78f73505b7e1 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:13.232297 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cdc5b-29e6-48ff-baab-78f73505b7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:13.232562 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cdc5b-29e6-48ff-baab-78f73505b7e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:13.245740 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302cdc5b-29e6-48ff-baab-78f73505b7e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:13.250972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-302cdc5b-29e6-48ff-baab-78f73505b7e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:13.250972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:13.250972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:13.250972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:13.250972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:13.250972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:13.250972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:13.250972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:13.250972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:13.250972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:13.250972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:13.250972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:13.250972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:13.250972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:13.250972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:13.250972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:13.251910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:13.251910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:13.251910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:13.251910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:13.251910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:13.251910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:13.251910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:13.251910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:13.251910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:13.251910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:13.251910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:13.251910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:13.251910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:13.251910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:13.251910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:13.251910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:13.253338 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:13.253338 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:13.253338 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:13.253338 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:13.253338 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:13.253338 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:13.253338 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:13.253338 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:13.253338 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:13.253338 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:13.253338 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:13.253338 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:13.253338 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 284/578] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:54:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:13.253338 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cdc5b-29e6-48ff-baab-78f73505b7e1 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:13.258105 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65146cd-efda-489c-8a6a-c202a3c05948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:13.258739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65146cd-efda-489c-8a6a-c202a3c05948 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:13.259700 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65146cd-efda-489c-8a6a-c202a3c05948 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:13.259700 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65146cd-efda-489c-8a6a-c202a3c05948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:13.271090 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65146cd-efda-489c-8a6a-c202a3c05948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:13.272553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65146cd-efda-489c-8a6a-c202a3c05948 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:13.286872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-e65146cd-efda-489c-8a6a-c202a3c05948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:13.286872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:13.286872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:13.286872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:13.286872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:13.286872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:13.286872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:13.286872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:13.286872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:13.286872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:13.286872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:13.286872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:13.286872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:13.286872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:13.286872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:13.286872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:13.287832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:13.287832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:13.287832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:13.287832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:13.287832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:13.287832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:13.287832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:13.287832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:13.287832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:13.287832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:13.287832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:13.287832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:13.287832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:13.287832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:13.287832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:13.287832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:13.287832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:13.288755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:13.288755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:13.288755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:13.288755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 295/579] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:13.288755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65146cd-efda-489c-8a6a-c202a3c05948 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:13.370252 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cdc5b-29e6-48ff-baab-78f73505b7e1 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:13.400625 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:13.402065 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65146cd-efda-489c-8a6a-c202a3c05948 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:13.407382 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d8938-6f11-4c35-ad48-7c31c9a4f26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:13.407987 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d8938-6f11-4c35-ad48-7c31c9a4f26d None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:13.408262 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d8938-6f11-4c35-ad48-7c31c9a4f26d None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:13.408525 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d8938-6f11-4c35-ad48-7c31c9a4f26d None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:13.409217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d8938-6f11-4c35-ad48-7c31c9a4f26d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:13.409513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d8938-6f11-4c35-ad48-7c31c9a4f26d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:13.411312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d8938-6f11-4c35-ad48-7c31c9a4f26d None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:13.426294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 285/580] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:13.427747 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d8938-6f11-4c35-ad48-7c31c9a4f26d None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:13.435128 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:13.446612 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f60d06-5678-4d88-a5e3-8f3b80a403ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:13.447484 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f60d06-5678-4d88-a5e3-8f3b80a403ca None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:13.448005 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f60d06-5678-4d88-a5e3-8f3b80a403ca None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:13.448673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f60d06-5678-4d88-a5e3-8f3b80a403ca None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:13.449846 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f60d06-5678-4d88-a5e3-8f3b80a403ca None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:13.450195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f60d06-5678-4d88-a5e3-8f3b80a403ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:13.451717 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f60d06-5678-4d88-a5e3-8f3b80a403ca None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:13.471921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 296/581] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Sep 14 01:54:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:13.472833 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f60d06-5678-4d88-a5e3-8f3b80a403ca None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:13.557304 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d8938-6f11-4c35-ad48-7c31c9a4f26d None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:13.582708 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f60d06-5678-4d88-a5e3-8f3b80a403ca None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:13.590938 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0c4f2-3e9a-4aad-a6fa-ec7522296ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:13.591538 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0c4f2-3e9a-4aad-a6fa-ec7522296ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:13.591811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0c4f2-3e9a-4aad-a6fa-ec7522296ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:13.592083 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0c4f2-3e9a-4aad-a6fa-ec7522296ed2 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:13.592735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0c4f2-3e9a-4aad-a6fa-ec7522296ed2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:13.593032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0c4f2-3e9a-4aad-a6fa-ec7522296ed2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:13.594439 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0c4f2-3e9a-4aad-a6fa-ec7522296ed2 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:13.602311 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 286/582] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 14 01:54:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:13.602906 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0c4f2-3e9a-4aad-a6fa-ec7522296ed2 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:13.627998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bf411-52be-4543-ab5b-45e3867e18d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:13.629035 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bf411-52be-4543-ab5b-45e3867e18d6 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:13.629391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bf411-52be-4543-ab5b-45e3867e18d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:13.629662 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bf411-52be-4543-ab5b-45e3867e18d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:13.630331 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580bf411-52be-4543-ab5b-45e3867e18d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:13.630631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580bf411-52be-4543-ab5b-45e3867e18d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:13.632247 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580bf411-52be-4543-ab5b-45e3867e18d6 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:13.645273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 297/583] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:13.646339 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bf411-52be-4543-ab5b-45e3867e18d6 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:13.669304 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:13.693283 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:13.720999 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0c4f2-3e9a-4aad-a6fa-ec7522296ed2 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:13.753259 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bf411-52be-4543-ab5b-45e3867e18d6 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:13.758023 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202bc920-5e0f-497a-9565-67b390ee6cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:13.758649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202bc920-5e0f-497a-9565-67b390ee6cdc None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:13.758925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202bc920-5e0f-497a-9565-67b390ee6cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:13.759191 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202bc920-5e0f-497a-9565-67b390ee6cdc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:13.766588 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202bc920-5e0f-497a-9565-67b390ee6cdc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:13.768065 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202bc920-5e0f-497a-9565-67b390ee6cdc None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:13.779420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-202bc920-5e0f-497a-9565-67b390ee6cdc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 01:54:13.779420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:13.779420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:13.779420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:13.779420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:13.779420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:13.779420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:13.779420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:13.779420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:13.779420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:13.779420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:13.779420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:13.779420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:13.779420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:13.779420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:13.779420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:13.780469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:13.780469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:13.780469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:13.780469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:13.780469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:13.780469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:13.780469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:13.780469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:13.780469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:13.780469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:13.780469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:13.780469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:13.780469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:13.780469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:13.780469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:13.780469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:13.780469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:13.780469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:13.782140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:13.782140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:13.782140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:13.782140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:13.782140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:13.782140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:13.782140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:13.782140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:13.782140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 01:54:13.782140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:13.782140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 287/584] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:54:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:13.782140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202bc920-5e0f-497a-9565-67b390ee6cdc None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:13.793116 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3fb4f-3cfd-49b8-83e5-eef72367efb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:13.793744 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3fb4f-3cfd-49b8-83e5-eef72367efb7 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:13.794037 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3fb4f-3cfd-49b8-83e5-eef72367efb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:13.794315 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3fb4f-3cfd-49b8-83e5-eef72367efb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:13.806117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3fb4f-3cfd-49b8-83e5-eef72367efb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:13.807569 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3fb4f-3cfd-49b8-83e5-eef72367efb7 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:13.821827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-08a3fb4f-3cfd-49b8-83e5-eef72367efb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:13.821827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:13.821827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:13.821827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:13.821827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:13.821827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:13.821827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:13.821827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:13.821827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:13.821827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:13.821827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:13.821827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:13.821827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:13.821827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:13.821827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:13.821827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:13.822848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:13.822848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:13.822848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:13.822848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:13.822848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:13.822848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:13.822848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:13.822848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:13.822848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:13.822848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:13.822848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:13.822848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:13.822848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:13.822848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:13.822848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:13.822848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:13.822848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:13.823782 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:13.823782 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:13.823782 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:13.823782 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 298/585] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:13.823782 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3fb4f-3cfd-49b8-83e5-eef72367efb7 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:13.889811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202bc920-5e0f-497a-9565-67b390ee6cdc None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:13.902612 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:13.924772 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94294266-646e-4382-bc59-430a22ba32c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:13.925399 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94294266-646e-4382-bc59-430a22ba32c8 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:13.925677 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94294266-646e-4382-bc59-430a22ba32c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:13.925951 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94294266-646e-4382-bc59-430a22ba32c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:13.926318 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:13.930558 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3fb4f-3cfd-49b8-83e5-eef72367efb7 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:13.938344 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94294266-646e-4382-bc59-430a22ba32c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:13.939829 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94294266-646e-4382-bc59-430a22ba32c8 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:13.955211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-94294266-646e-4382-bc59-430a22ba32c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:13.955211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:13.955211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:13.955211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:13.955211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:13.955211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:13.955211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:13.955211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:13.955211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:13.955211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:13.955211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:13.955211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:13.955211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:13.955211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:13.955211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:13.955211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:13.956268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:13.956268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:13.956268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:13.956268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:13.956268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:13.956268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:13.956268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:13.956268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:13.956268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:13.956268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:13.956268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:13.956268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:13.956268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:13.956268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:13.956268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:13.956268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:13.956268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:13.957226 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:13.957226 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:13.957226 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:13.957226 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 288/586] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:13.957226 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94294266-646e-4382-bc59-430a22ba32c8 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:13.970605 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b872e-ba43-4c98-a6cb-77ca9e402cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:13.971891 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b872e-ba43-4c98-a6cb-77ca9e402cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:13.972090 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b872e-ba43-4c98-a6cb-77ca9e402cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:13.972357 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b872e-ba43-4c98-a6cb-77ca9e402cf6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:13.979303 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b872e-ba43-4c98-a6cb-77ca9e402cf6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:13.980753 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b872e-ba43-4c98-a6cb-77ca9e402cf6 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:13.991730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-291b872e-ba43-4c98-a6cb-77ca9e402cf6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:13.991730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:13.991730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:13.991730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:13.991730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:13.991730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:13.991730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:13.991730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:13.991730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:13.991730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:13.991730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:13.991730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:13.991730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:13.991730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:13.991730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:13.992671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:13.992671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:13.992671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:13.992671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:13.992671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:13.992671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:13.992671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:13.992671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:13.992671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:13.992671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:13.992671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:13.992671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:13.992671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:13.992671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:13.992671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:13.992671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:13.992671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:13.993598 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:13.993598 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:13.993598 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:13.993598 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:13.993598 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:13.993598 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:13.993598 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:13.993598 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:13.993598 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 299/587] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:13.993598 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b872e-ba43-4c98-a6cb-77ca9e402cf6 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:14.062538 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94294266-646e-4382-bc59-430a22ba32c8 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:14.098102 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43822417-4dc2-43ef-afc4-8a593c654b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:14.098765 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43822417-4dc2-43ef-afc4-8a593c654b96 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:14.099052 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43822417-4dc2-43ef-afc4-8a593c654b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:14.099327 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43822417-4dc2-43ef-afc4-8a593c654b96 None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:14.100048 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43822417-4dc2-43ef-afc4-8a593c654b96 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:14.100351 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43822417-4dc2-43ef-afc4-8a593c654b96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:14.101799 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43822417-4dc2-43ef-afc4-8a593c654b96 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:14.102380 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b872e-ba43-4c98-a6cb-77ca9e402cf6 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:14.120002 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 289/588] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Sep 14 01:54:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:14.120807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43822417-4dc2-43ef-afc4-8a593c654b96 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:14.142462 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b786ba0-9f23-45b3-9530-b94d514d4f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:14.143517 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b786ba0-9f23-45b3-9530-b94d514d4f85 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:14.143795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b786ba0-9f23-45b3-9530-b94d514d4f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:14.144066 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b786ba0-9f23-45b3-9530-b94d514d4f85 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:14.144693 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b786ba0-9f23-45b3-9530-b94d514d4f85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:14.144990 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b786ba0-9f23-45b3-9530-b94d514d4f85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:14.146378 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b786ba0-9f23-45b3-9530-b94d514d4f85 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:14.147919 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:14.162713 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 300/589] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:14.162713 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b786ba0-9f23-45b3-9530-b94d514d4f85 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:14.180589 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:14.234980 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43822417-4dc2-43ef-afc4-8a593c654b96 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:14.263292 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b786ba0-9f23-45b3-9530-b94d514d4f85 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:14.271420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f870536-31bd-4609-b188-3d460b913bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:14.272020 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f870536-31bd-4609-b188-3d460b913bdb None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:14.272392 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f870536-31bd-4609-b188-3d460b913bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:14.272668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f870536-31bd-4609-b188-3d460b913bdb None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:14.273349 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f870536-31bd-4609-b188-3d460b913bdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:14.273653 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f870536-31bd-4609-b188-3d460b913bdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:14.275078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f870536-31bd-4609-b188-3d460b913bdb None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:14.288617 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 290/590] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:14.289850 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f870536-31bd-4609-b188-3d460b913bdb None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:14.305648 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f5d8b-26dd-4a05-8153-707127f38be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:14.306749 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f5d8b-26dd-4a05-8153-707127f38be4 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:14.307027 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f5d8b-26dd-4a05-8153-707127f38be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:14.307440 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f5d8b-26dd-4a05-8153-707127f38be4 None admin] PATH_INFO: `/v3/projects` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:14.308127 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f5d8b-26dd-4a05-8153-707127f38be4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:14.308428 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f5d8b-26dd-4a05-8153-707127f38be4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:14.309845 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f5d8b-26dd-4a05-8153-707127f38be4 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:14.321544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 301/591] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:14.322182 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f5d8b-26dd-4a05-8153-707127f38be4 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:14.384829 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:14.404987 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f870536-31bd-4609-b188-3d460b913bdb None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:14.413095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:14.437345 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f5d8b-26dd-4a05-8153-707127f38be4 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:14.440657 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95713173-2b12-4a21-a4b2-c3c3d1806463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:14.441348 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95713173-2b12-4a21-a4b2-c3c3d1806463 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:14.441626 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95713173-2b12-4a21-a4b2-c3c3d1806463 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:14.441895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95713173-2b12-4a21-a4b2-c3c3d1806463 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:14.453586 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95713173-2b12-4a21-a4b2-c3c3d1806463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:14.455084 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95713173-2b12-4a21-a4b2-c3c3d1806463 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:14.469162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-95713173-2b12-4a21-a4b2-c3c3d1806463 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:14.469162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:14.469162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:14.469162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:14.469162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:14.469162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:14.469162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:14.469162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:14.469162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:14.469162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:14.469162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:14.469162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:14.469162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:14.469162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:14.469162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:14.469162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:14.470358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:14.470358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:14.470358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:14.470358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:14.470358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:14.470358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:14.470358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:14.470358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:14.470358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:14.470358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:14.470358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:14.470358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:14.470358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:14.470358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:14.470358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:14.470358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:14.470358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:14.471841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:14.471841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:14.471841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:14.471841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 291/592] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:14.472232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95713173-2b12-4a21-a4b2-c3c3d1806463 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:14.480080 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb3590-1625-461c-933a-37b38106b28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:14.480721 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb3590-1625-461c-933a-37b38106b28f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:14.480996 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb3590-1625-461c-933a-37b38106b28f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:14.481261 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb3590-1625-461c-933a-37b38106b28f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:14.489489 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb3590-1625-461c-933a-37b38106b28f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:14.490970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb3590-1625-461c-933a-37b38106b28f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:14.502238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-03eb3590-1625-461c-933a-37b38106b28f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:14.502238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:14.502238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:14.502238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:14.502238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:14.502238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:14.502238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:14.502238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:14.502238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:14.502238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:14.502238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:14.502238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:14.502238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:14.502238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:14.502238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:14.503193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:14.503193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:14.503193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:14.503193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:14.503193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:14.503193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:14.503193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:14.503193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:14.503193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:14.503193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:14.503193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:14.503193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:14.503193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:14.503193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:14.503193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:14.503193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:14.503193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:14.504115 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:14.504115 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:14.504115 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:14.504115 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:14.504115 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:14.504115 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:14.504115 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:14.504115 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:14.504115 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 302/593] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:14.504115 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb3590-1625-461c-933a-37b38106b28f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:14.586396 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95713173-2b12-4a21-a4b2-c3c3d1806463 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:14.621574 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb3590-1625-461c-933a-37b38106b28f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:14.622372 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766595a-ee39-4023-b04a-258b514f2163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:14.623260 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766595a-ee39-4023-b04a-258b514f2163 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:14.623769 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766595a-ee39-4023-b04a-258b514f2163 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:14.624160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766595a-ee39-4023-b04a-258b514f2163 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:14.626276 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:14.634127 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766595a-ee39-4023-b04a-258b514f2163 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:14.635748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766595a-ee39-4023-b04a-258b514f2163 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:14.647949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-d766595a-ee39-4023-b04a-258b514f2163 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 01:54:14.647949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:14.647949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:14.647949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:14.647949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:14.647949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:14.647949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:14.647949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:14.647949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:14.647949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:14.647949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:14.647949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:14.647949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:14.647949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:14.647949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:14.647949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:14.649118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:14.649118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:14.649118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:14.649118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:14.649118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:14.649118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:14.649118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:14.649118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:14.649118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:14.649118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:14.649118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:14.649118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:14.649118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:14.649118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:14.649118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:14.649118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:14.649118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:14.649118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:14.650302 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:14.650302 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:14.650302 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:14.650302 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:14.650302 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:14.650302 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:14.650302 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:14.650302 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:14.650302 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 01:54:14.650302 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:14.650302 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 292/594] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:54:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:14.650302 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766595a-ee39-4023-b04a-258b514f2163 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:14.666562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae7e19-b196-42f4-9bfc-9d944a5a1604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:14.667200 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae7e19-b196-42f4-9bfc-9d944a5a1604 None admin] REQUEST_METHOD: `PUT` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:14.667430 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae7e19-b196-42f4-9bfc-9d944a5a1604 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:14.667700 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae7e19-b196-42f4-9bfc-9d944a5a1604 None admin] PATH_INFO: `/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/f721a3a4eca941989685448a87bb9a19/roles/e962a2a280224f9fbcd0fdc02262993d` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:14.677365 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:14.685084 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae7e19-b196-42f4-9bfc-9d944a5a1604 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e04b16dba1c4de79dc6b1a5e16ed907, user_id=f721a3a4eca941989685448a87bb9a19, role_id=e962a2a280224f9fbcd0fdc02262993d)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:14.686626 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae7e19-b196-42f4-9bfc-9d944a5a1604 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:14.700098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 303/595] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:54:14 2022] PUT /identity/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/f721a3a4eca941989685448a87bb9a19/roles/e962a2a280224f9fbcd0fdc02262993d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:54:14.700801 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae7e19-b196-42f4-9bfc-9d944a5a1604 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:14.787019 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766595a-ee39-4023-b04a-258b514f2163 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:14.827716 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04069ae2-97ef-4962-a082-7210fc1cd368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:14.827963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04069ae2-97ef-4962-a082-7210fc1cd368 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:14.828190 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04069ae2-97ef-4962-a082-7210fc1cd368 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:14.828432 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04069ae2-97ef-4962-a082-7210fc1cd368 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:14.829127 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04069ae2-97ef-4962-a082-7210fc1cd368 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:14.829604 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04069ae2-97ef-4962-a082-7210fc1cd368 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:14.831642 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04069ae2-97ef-4962-a082-7210fc1cd368 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:14.844887 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 293/596] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:14.846718 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae7e19-b196-42f4-9bfc-9d944a5a1604 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:14.851021 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04069ae2-97ef-4962-a082-7210fc1cd368 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:14.895718 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f49d28-9e60-42ee-b164-6bce1d138f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:14.896041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f49d28-9e60-42ee-b164-6bce1d138f35 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:14.896321 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f49d28-9e60-42ee-b164-6bce1d138f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:14.896594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f49d28-9e60-42ee-b164-6bce1d138f35 None admin] PATH_INFO: `/v3/projects` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:14.897289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f49d28-9e60-42ee-b164-6bce1d138f35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:14.897598 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f49d28-9e60-42ee-b164-6bce1d138f35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:14.899145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f49d28-9e60-42ee-b164-6bce1d138f35 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:14.913003 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:14.913479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 304/597] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:14.915514 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c6c31-a60a-41c8-b2cd-acba14acf382 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:14.915798 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c6c31-a60a-41c8-b2cd-acba14acf382 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:14.916108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c6c31-a60a-41c8-b2cd-acba14acf382 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:14.917527 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 305/598] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:14.918083 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c6c31-a60a-41c8-b2cd-acba14acf382 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:14.964514 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04069ae2-97ef-4962-a082-7210fc1cd368 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:14.973700 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:15.005014 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b78d5-3e81-486a-8c3a-91a6e85cb5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:15.005620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b78d5-3e81-486a-8c3a-91a6e85cb5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:15.006177 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b78d5-3e81-486a-8c3a-91a6e85cb5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:15.006446 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b78d5-3e81-486a-8c3a-91a6e85cb5e9 None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:15.007160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b78d5-3e81-486a-8c3a-91a6e85cb5e9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:15.007655 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b78d5-3e81-486a-8c3a-91a6e85cb5e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:15.009095 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b78d5-3e81-486a-8c3a-91a6e85cb5e9 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:15.026209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 294/599] 10.222.0.15 () {56 vars in 1139 bytes} [Wed Sep 14 01:54:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:15.026209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b78d5-3e81-486a-8c3a-91a6e85cb5e9 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:15.037928 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c6c31-a60a-41c8-b2cd-acba14acf382 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:15.073791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd956e1-c4ca-41c0-bfcd-f0b4c7ce2751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:15.074468 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd956e1-c4ca-41c0-bfcd-f0b4c7ce2751 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:15.074744 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd956e1-c4ca-41c0-bfcd-f0b4c7ce2751 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:15.075031 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd956e1-c4ca-41c0-bfcd-f0b4c7ce2751 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:15.081845 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd956e1-c4ca-41c0-bfcd-f0b4c7ce2751 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:15.083366 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd956e1-c4ca-41c0-bfcd-f0b4c7ce2751 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:15.094724 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-9dd956e1-c4ca-41c0-bfcd-f0b4c7ce2751 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:15.094724 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:15.094724 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:15.094724 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:15.094724 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:15.094724 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:15.094724 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:15.094724 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:15.094724 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:15.094724 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:15.094724 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:15.094724 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:15.094724 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:15.094724 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:15.094724 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:15.095631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:15.095631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:15.095631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:15.095631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:15.095631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:15.095631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:15.095631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:15.095631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:15.095631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:15.095631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:15.095631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:15.095631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:15.095631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:15.095631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:15.095631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:15.095631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:15.095631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:15.096630 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:15.096630 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:15.096630 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:15.096630 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:15.096630 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:15.096630 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:15.096630 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:15.096630 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:15.096630 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 306/600] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:15.097610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a06613-0b93-4fcc-a3fd-5703973e4318 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:15.097709 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a06613-0b93-4fcc-a3fd-5703973e4318 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:15.097923 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a06613-0b93-4fcc-a3fd-5703973e4318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:15.116009 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-f7a06613-0b93-4fcc-a3fd-5703973e4318 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:15.117207 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-f7a06613-0b93-4fcc-a3fd-5703973e4318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:15.138486 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b78d5-3e81-486a-8c3a-91a6e85cb5e9 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:15.152431 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:15.175578 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6c2d4-1a67-4a22-b022-c3a829e55c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:15.176163 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6c2d4-1a67-4a22-b022-c3a829e55c3f None admin] REQUEST_METHOD: `PUT` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:15.176438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6c2d4-1a67-4a22-b022-c3a829e55c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:15.176708 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6c2d4-1a67-4a22-b022-c3a829e55c3f None admin] PATH_INFO: `/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/5a5191afa5f246269d5b4a60972332a6/roles/e962a2a280224f9fbcd0fdc02262993d` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:15.189400 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 307/601] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:54:15.190886 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a06613-0b93-4fcc-a3fd-5703973e4318 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:15.193282 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6c2d4-1a67-4a22-b022-c3a829e55c3f None admin] RBAC: Authorizing `identity:create_grant(project_id=9e04b16dba1c4de79dc6b1a5e16ed907, user_id=5a5191afa5f246269d5b4a60972332a6, role_id=e962a2a280224f9fbcd0fdc02262993d)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:15.195050 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6c2d4-1a67-4a22-b022-c3a829e55c3f None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:15.222013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 295/602] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:54:15 2022] PUT /identity/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/5a5191afa5f246269d5b4a60972332a6/roles/e962a2a280224f9fbcd0fdc02262993d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:54:15.222013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6c2d4-1a67-4a22-b022-c3a829e55c3f None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:15.281550 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:15.406923 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a06613-0b93-4fcc-a3fd-5703973e4318 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:15.452210 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6c2d4-1a67-4a22-b022-c3a829e55c3f None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:15.456500 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a8642-48ba-4de2-85a2-57cd36cc5382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:15.457371 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a8642-48ba-4de2-85a2-57cd36cc5382 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:15.457835 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a8642-48ba-4de2-85a2-57cd36cc5382 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:15.458357 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a8642-48ba-4de2-85a2-57cd36cc5382 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:15.472581 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a8642-48ba-4de2-85a2-57cd36cc5382 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:15.474436 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a8642-48ba-4de2-85a2-57cd36cc5382 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:15.490906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-446a8642-48ba-4de2-85a2-57cd36cc5382 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:15.490906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:15.490906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:15.490906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:15.490906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:15.490906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:15.490906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:15.490906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:15.490906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:15.490906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:15.490906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:15.490906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:15.490906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:15.490906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:15.490906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:15.490906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:15.492129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:15.492129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:15.492129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:15.492129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:15.492129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:15.492129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:15.492129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:15.492129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:15.492129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:15.492129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:15.492129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:15.492129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:15.492129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:15.492129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:15.492129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:15.492129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:15.492129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:15.493256 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:15.493256 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:15.493256 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:15.501673 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3538589-1912-4bb3-a754-4ef0d9e37e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:15.501673 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3538589-1912-4bb3-a754-4ef0d9e37e5c None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:15.501968 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 308/603] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:15.502094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3538589-1912-4bb3-a754-4ef0d9e37e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:15.502094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3538589-1912-4bb3-a754-4ef0d9e37e5c None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:15.502321 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a8642-48ba-4de2-85a2-57cd36cc5382 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:15.502533 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3538589-1912-4bb3-a754-4ef0d9e37e5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:15.502533 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3538589-1912-4bb3-a754-4ef0d9e37e5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:15.502533 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3538589-1912-4bb3-a754-4ef0d9e37e5c None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:15.512926 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 296/604] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:15.513848 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3538589-1912-4bb3-a754-4ef0d9e37e5c None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:15.547412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:15.611298 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:15.624549 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a8642-48ba-4de2-85a2-57cd36cc5382 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:15.633412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3538589-1912-4bb3-a754-4ef0d9e37e5c None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:15.662913 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd9fe2-83f3-4014-9421-3ed3dc9742a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:15.663550 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd9fe2-83f3-4014-9421-3ed3dc9742a7 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:15.663824 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd9fe2-83f3-4014-9421-3ed3dc9742a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:15.664149 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd9fe2-83f3-4014-9421-3ed3dc9742a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:15.670384 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c4295-4c2e-4761-aab5-7903a3044e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:15.670991 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c4295-4c2e-4761-aab5-7903a3044e94 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:15.671373 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c4295-4c2e-4761-aab5-7903a3044e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:15.671646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c4295-4c2e-4761-aab5-7903a3044e94 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:15.672382 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c4295-4c2e-4761-aab5-7903a3044e94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:15.672684 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c4295-4c2e-4761-aab5-7903a3044e94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:15.674138 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c4295-4c2e-4761-aab5-7903a3044e94 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:15.675380 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd9fe2-83f3-4014-9421-3ed3dc9742a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:15.680926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-15bd9fe2-83f3-4014-9421-3ed3dc9742a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:15.680926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:15.680926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:15.680926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:15.680926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:15.680926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:15.680926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:15.680926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:15.680926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:15.680926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:15.680926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:15.680926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:15.680926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:15.680926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:15.680926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:15.680926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:15.683257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:15.683257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:15.683257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:15.683257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:15.683257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:15.683257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:15.683257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:15.683257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:15.683257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:15.683257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:15.683257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:15.683257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:15.683257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:15.683257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:15.683257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:15.683257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:15.685379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:15.685379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:15.685379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:15.685379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:15.685379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:15.685379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:15.685379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:15.685379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:15.685379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:15.685379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:15.685379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:15.685379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:15.685379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 309/605] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:54:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:15.685379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd9fe2-83f3-4014-9421-3ed3dc9742a7 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:15.688661 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 297/606] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:15.688661 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c4295-4c2e-4761-aab5-7903a3044e94 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:15.774090 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:15.793457 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c4295-4c2e-4761-aab5-7903a3044e94 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:15.793745 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd9fe2-83f3-4014-9421-3ed3dc9742a7 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:15.829537 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52dab2-2aed-4ab6-8114-8830b538a934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:15.829886 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30ec10-2d57-49f2-8337-a09368a969da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:15.830228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52dab2-2aed-4ab6-8114-8830b538a934 None admin] REQUEST_METHOD: `PUT` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:15.830917 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30ec10-2d57-49f2-8337-a09368a969da None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:15.831078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52dab2-2aed-4ab6-8114-8830b538a934 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:15.831242 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30ec10-2d57-49f2-8337-a09368a969da None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:15.831393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52dab2-2aed-4ab6-8114-8830b538a934 None admin] PATH_INFO: `/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/c6d70ad7054e40c48f8751a3d4fed787/roles/e962a2a280224f9fbcd0fdc02262993d` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:15.831567 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30ec10-2d57-49f2-8337-a09368a969da None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:15.834007 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30ec10-2d57-49f2-8337-a09368a969da None admin] RBAC: Adding query filter params (name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:15.834007 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30ec10-2d57-49f2-8337-a09368a969da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:15.834007 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30ec10-2d57-49f2-8337-a09368a969da None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:15.842653 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 298/607] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 14 01:54:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:15.843552 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30ec10-2d57-49f2-8337-a09368a969da None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:15.846790 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52dab2-2aed-4ab6-8114-8830b538a934 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e04b16dba1c4de79dc6b1a5e16ed907, user_id=c6d70ad7054e40c48f8751a3d4fed787, role_id=e962a2a280224f9fbcd0fdc02262993d)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:15.848281 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52dab2-2aed-4ab6-8114-8830b538a934 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:15.859341 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:15.870716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 310/608] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:54:15 2022] PUT /identity/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/c6d70ad7054e40c48f8751a3d4fed787/roles/e962a2a280224f9fbcd0fdc02262993d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:54:15.872338 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52dab2-2aed-4ab6-8114-8830b538a934 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:15.987059 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30ec10-2d57-49f2-8337-a09368a969da None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:16.056449 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e6de4-7fc9-4d02-94f8-160ac9ef13b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:16.057412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e6de4-7fc9-4d02-94f8-160ac9ef13b4 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:16.057941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e6de4-7fc9-4d02-94f8-160ac9ef13b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:16.058432 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e6de4-7fc9-4d02-94f8-160ac9ef13b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:16.067068 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e6de4-7fc9-4d02-94f8-160ac9ef13b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:16.068880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e6de4-7fc9-4d02-94f8-160ac9ef13b4 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:16.081793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-b07e6de4-7fc9-4d02-94f8-160ac9ef13b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:16.081793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:16.081793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:16.081793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:16.081793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:16.081793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:16.081793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:16.081793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:16.081793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:16.081793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:16.081793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:16.081793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:16.081793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:16.081793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:16.081793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:16.082903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:16.082903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:16.082903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:16.082903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:16.082903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:16.082903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:16.082903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:16.082903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:16.082903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:16.082903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:16.082903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:16.082903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:16.082903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:16.082903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:16.082903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:16.082903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:16.082903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:16.083821 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:16.083821 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:16.083821 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:16.083821 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:16.083821 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:16.083821 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:16.083821 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:16.083821 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:16.085972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 299/609] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:16.091732 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e6de4-7fc9-4d02-94f8-160ac9ef13b4 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:16.098583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:16.109574 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52dab2-2aed-4ab6-8114-8830b538a934 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:16.154606 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755fbee-c866-468d-9c53-cd651addb16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:16.155245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755fbee-c866-468d-9c53-cd651addb16a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:16.156354 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755fbee-c866-468d-9c53-cd651addb16a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:16.156354 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755fbee-c866-468d-9c53-cd651addb16a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:16.170667 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755fbee-c866-468d-9c53-cd651addb16a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:16.173135 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755fbee-c866-468d-9c53-cd651addb16a None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:16.193434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-b755fbee-c866-468d-9c53-cd651addb16a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:16.193434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:16.193434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:16.193434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:16.193434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:16.193434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:16.193434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:16.193434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:16.193434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:16.193434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:16.193434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:16.193434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:16.193434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:16.193434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:16.193434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:16.193434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:16.194715 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:16.194715 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:16.194715 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:16.194715 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:16.194715 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:16.194715 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:16.194715 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:16.194715 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:16.194715 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:16.194715 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:16.194715 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:16.194715 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:16.194715 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:16.194715 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:16.194715 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:16.194715 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:16.194715 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:16.195785 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:16.195785 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:16.195785 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:16.195785 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 311/610] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:16.203244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755fbee-c866-468d-9c53-cd651addb16a None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:16.262266 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e6de4-7fc9-4d02-94f8-160ac9ef13b4 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:16.302892 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:16.335390 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597cfc2-f603-4883-a2c5-697afbc56c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:16.336347 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597cfc2-f603-4883-a2c5-697afbc56c56 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:16.336843 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597cfc2-f603-4883-a2c5-697afbc56c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:16.337324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597cfc2-f603-4883-a2c5-697afbc56c56 None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:16.338251 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597cfc2-f603-4883-a2c5-697afbc56c56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:16.338769 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597cfc2-f603-4883-a2c5-697afbc56c56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:16.340402 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597cfc2-f603-4883-a2c5-697afbc56c56 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:16.356308 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755fbee-c866-468d-9c53-cd651addb16a None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:16.362937 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 300/611] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:54:16.370916 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597cfc2-f603-4883-a2c5-697afbc56c56 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:16.395052 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad378d-66ae-4656-bbb3-8d281636080a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:16.395666 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad378d-66ae-4656-bbb3-8d281636080a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:16.395943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad378d-66ae-4656-bbb3-8d281636080a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:16.402027 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad378d-66ae-4656-bbb3-8d281636080a None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:16.402027 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad378d-66ae-4656-bbb3-8d281636080a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:16.402027 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad378d-66ae-4656-bbb3-8d281636080a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:16.402027 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad378d-66ae-4656-bbb3-8d281636080a None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:16.411822 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 312/612] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:16.416679 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad378d-66ae-4656-bbb3-8d281636080a None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:16.507055 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:16.527772 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad378d-66ae-4656-bbb3-8d281636080a None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:16.542057 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:16.567129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b091d00-24e6-4b1f-81b5-599a04108f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:16.567872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b091d00-24e6-4b1f-81b5-599a04108f71 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:16.568153 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b091d00-24e6-4b1f-81b5-599a04108f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:16.568422 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b091d00-24e6-4b1f-81b5-599a04108f71 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:16.576307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b091d00-24e6-4b1f-81b5-599a04108f71 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:16.577756 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b091d00-24e6-4b1f-81b5-599a04108f71 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:16.590266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-3b091d00-24e6-4b1f-81b5-599a04108f71 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:54:16.590266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:16.590266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:16.590266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:16.590266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:16.590266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:16.590266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:16.590266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:16.590266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:16.590266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:16.590266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:16.590266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:16.590266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:16.590266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:16.590266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:16.590266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:16.591241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:16.591241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:16.591241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:16.591241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:16.591241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:16.591241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:16.591241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:16.591241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:16.591241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:16.591241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:16.591241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:16.591241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:16.591241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:16.591241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:16.591241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:16.591241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:16.591241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:16.591241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:16.592433 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:16.592433 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:16.592433 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:16.592433 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:16.592433 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:16.592433 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:16.592433 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:16.592433 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:16.592433 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:54:16.592433 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:16.592433 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 313/613] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:54:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:16.600042 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b091d00-24e6-4b1f-81b5-599a04108f71 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:16.603255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597cfc2-f603-4883-a2c5-697afbc56c56 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:16.675693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec089b5-7e8f-4761-a094-2ff28c10c487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:16.676368 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec089b5-7e8f-4761-a094-2ff28c10c487 None admin] REQUEST_METHOD: `PUT` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:16.676648 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec089b5-7e8f-4761-a094-2ff28c10c487 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:16.676922 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec089b5-7e8f-4761-a094-2ff28c10c487 None admin] PATH_INFO: `/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/bf86028f85af444380293d2387928eda/roles/e962a2a280224f9fbcd0fdc02262993d` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:16.709595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec089b5-7e8f-4761-a094-2ff28c10c487 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e04b16dba1c4de79dc6b1a5e16ed907, user_id=bf86028f85af444380293d2387928eda, role_id=e962a2a280224f9fbcd0fdc02262993d)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:16.712252 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec089b5-7e8f-4761-a094-2ff28c10c487 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:16.739939 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 301/614] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:54:16 2022] PUT /identity/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/bf86028f85af444380293d2387928eda/roles/e962a2a280224f9fbcd0fdc02262993d => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:54:16.785048 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b091d00-24e6-4b1f-81b5-599a04108f71 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:16.821459 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02830f-bcc0-4688-9cf4-6fe5d2716c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:16.822129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02830f-bcc0-4688-9cf4-6fe5d2716c97 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:16.822452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02830f-bcc0-4688-9cf4-6fe5d2716c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:16.822728 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02830f-bcc0-4688-9cf4-6fe5d2716c97 None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:16.823512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02830f-bcc0-4688-9cf4-6fe5d2716c97 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:16.823812 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02830f-bcc0-4688-9cf4-6fe5d2716c97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:16.825376 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02830f-bcc0-4688-9cf4-6fe5d2716c97 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:16.845893 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 314/615] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Sep 14 01:54:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:16.851777 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec089b5-7e8f-4761-a094-2ff28c10c487 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:17.009706 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec089b5-7e8f-4761-a094-2ff28c10c487 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:17.042420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:17.056339 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc1b77-9d47-4381-b164-36fc25f90879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:17.056996 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc1b77-9d47-4381-b164-36fc25f90879 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:17.057268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc1b77-9d47-4381-b164-36fc25f90879 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:17.057543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc1b77-9d47-4381-b164-36fc25f90879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:17.069216 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc1b77-9d47-4381-b164-36fc25f90879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:17.072417 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc1b77-9d47-4381-b164-36fc25f90879 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:17.086905 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d44db1-46fc-4b03-89e2-bde826941df1 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:17.087449 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d44db1-46fc-4b03-89e2-bde826941df1 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:17.087449 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d44db1-46fc-4b03-89e2-bde826941df1 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:17.088748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 315/616] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:54:17.091561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-a8fc1b77-9d47-4381-b164-36fc25f90879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:17.091561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:17.091561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:17.091561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:17.091561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:17.091561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:17.091561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:17.091561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:17.091561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:17.091561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:17.091561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:17.091561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:17.091561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:17.091561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:17.091561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:17.091561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:17.093544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:17.093544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:17.093544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:17.093544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:17.093544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:17.093544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:17.093544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:17.093544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:17.093544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:17.093544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:17.093544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:17.093544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:17.093544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:17.093544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:17.093544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:17.093544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:17.093544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:17.095541 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:17.095541 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:17.095541 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:17.095541 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 302/617] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:17.096107 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e9f07-97c5-40ae-b76e-af3d777b8a7b None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:17.096107 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e9f07-97c5-40ae-b76e-af3d777b8a7b None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:17.096107 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e9f07-97c5-40ae-b76e-af3d777b8a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:17.098469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc1b77-9d47-4381-b164-36fc25f90879 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:17.115069 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-888e9f07-97c5-40ae-b76e-af3d777b8a7b None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:17.116422 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-888e9f07-97c5-40ae-b76e-af3d777b8a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:17.152545 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:17.224074 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 316/618] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:54:17.245906 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc1b77-9d47-4381-b164-36fc25f90879 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:17.298270 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e074f-eaa6-43a4-a4f7-15f609bfbe7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:17.299538 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e074f-eaa6-43a4-a4f7-15f609bfbe7d None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:17.299707 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e074f-eaa6-43a4-a4f7-15f609bfbe7d None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:17.299913 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e074f-eaa6-43a4-a4f7-15f609bfbe7d None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:17.300549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e074f-eaa6-43a4-a4f7-15f609bfbe7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:17.300837 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e074f-eaa6-43a4-a4f7-15f609bfbe7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:17.302222 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e074f-eaa6-43a4-a4f7-15f609bfbe7d None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:17.317605 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 303/619] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:17.322713 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e9f07-97c5-40ae-b76e-af3d777b8a7b None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:17.453078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e9f07-97c5-40ae-b76e-af3d777b8a7b None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:17.493596 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db8952-8608-43e7-9624-c35e0ea2a1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:17.494255 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db8952-8608-43e7-9624-c35e0ea2a1dc None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:17.494529 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db8952-8608-43e7-9624-c35e0ea2a1dc None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:17.494793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db8952-8608-43e7-9624-c35e0ea2a1dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:17.501955 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db8952-8608-43e7-9624-c35e0ea2a1dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:17.503967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db8952-8608-43e7-9624-c35e0ea2a1dc None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:17.525356 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-b3db8952-8608-43e7-9624-c35e0ea2a1dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:17.525356 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:17.525356 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:17.525356 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:17.525356 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:17.525356 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:17.525356 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:17.525356 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:17.525356 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:17.525356 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:17.525356 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:17.525356 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:17.525356 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:17.525356 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:17.525356 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:17.527681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:17.527681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:17.527681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:17.527681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:17.527681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:17.527681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:17.527681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:17.527681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:17.527681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:17.527681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:17.527681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:17.527681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:17.527681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:17.527681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:17.527681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:17.527681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:17.527681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:17.529814 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:17.529814 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:17.529814 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:17.529814 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:17.529814 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:17.529814 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:17.529814 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:17.529814 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:17.529814 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 317/620] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:17.534381 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e074f-eaa6-43a4-a4f7-15f609bfbe7d None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:17.569306 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:17.614663 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db8952-8608-43e7-9624-c35e0ea2a1dc None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:17.648838 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e074f-eaa6-43a4-a4f7-15f609bfbe7d None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:17.654358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:17.685106 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd482f-09d3-42ca-9263-b044bbbee5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:17.685764 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd482f-09d3-42ca-9263-b044bbbee5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:17.686066 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd482f-09d3-42ca-9263-b044bbbee5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:17.686336 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd482f-09d3-42ca-9263-b044bbbee5c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:17.688328 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd482f-09d3-42ca-9263-b044bbbee5c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:17.688328 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd482f-09d3-42ca-9263-b044bbbee5c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:17.690331 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd482f-09d3-42ca-9263-b044bbbee5c4 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:17.701570 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 304/621] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:17.707087 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd482f-09d3-42ca-9263-b044bbbee5c4 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:17.834531 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db8952-8608-43e7-9624-c35e0ea2a1dc None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:17.837752 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:17.854524 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd482f-09d3-42ca-9263-b044bbbee5c4 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:17.891448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301c1c7-6ad2-49cc-b400-7a276a9e798e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:17.892083 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301c1c7-6ad2-49cc-b400-7a276a9e798e None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:17.892360 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301c1c7-6ad2-49cc-b400-7a276a9e798e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:17.892743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301c1c7-6ad2-49cc-b400-7a276a9e798e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:17.903097 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdbc920-567a-4d31-ac9d-2f3aa6c5840f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:17.903961 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdbc920-567a-4d31-ac9d-2f3aa6c5840f None admin] REQUEST_METHOD: `PUT` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:17.904237 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdbc920-567a-4d31-ac9d-2f3aa6c5840f None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:17.904351 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301c1c7-6ad2-49cc-b400-7a276a9e798e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:17.904529 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdbc920-567a-4d31-ac9d-2f3aa6c5840f None admin] PATH_INFO: `/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/21f481c1f60043e9b0e25ba49b30715f/roles/e962a2a280224f9fbcd0fdc02262993d` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:17.909844 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-7301c1c7-6ad2-49cc-b400-7a276a9e798e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:17.909844 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:17.909844 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:17.909844 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:17.909844 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:17.909844 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:17.909844 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:17.909844 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:17.909844 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:17.909844 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:17.909844 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:17.909844 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:17.909844 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:17.909844 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:17.909844 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:17.909844 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:17.910866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:17.910866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:17.910866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:17.910866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:17.910866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:17.910866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:17.910866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:17.910866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:17.910866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:17.910866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:17.910866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:17.910866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:17.910866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:17.910866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:17.910866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:17.910866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:17.912091 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:17.912091 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:17.912091 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:17.912091 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:17.912091 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:17.912091 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:17.912091 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:17.912091 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:17.912091 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:17.912091 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:17.912091 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:17.912091 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:17.912091 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 318/622] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:54:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:17.912829 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec1367-479c-425d-8d0a-d16e2db32a8d None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:17.912829 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec1367-479c-425d-8d0a-d16e2db32a8d None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:17.913231 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec1367-479c-425d-8d0a-d16e2db32a8d None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:17.915522 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 319/623] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:17.919658 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded744a-befe-4e6b-a468-1578e7598449 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:17.919935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded744a-befe-4e6b-a468-1578e7598449 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:17.920208 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded744a-befe-4e6b-a468-1578e7598449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:17.923592 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdbc920-567a-4d31-ac9d-2f3aa6c5840f None admin] RBAC: Authorizing `identity:create_grant(project_id=9e04b16dba1c4de79dc6b1a5e16ed907, user_id=21f481c1f60043e9b0e25ba49b30715f, role_id=e962a2a280224f9fbcd0fdc02262993d)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:17.925688 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdbc920-567a-4d31-ac9d-2f3aa6c5840f None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:17.940824 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-2ded744a-befe-4e6b-a468-1578e7598449 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:17.941669 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 305/624] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:54:17 2022] PUT /identity/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/21f481c1f60043e9b0e25ba49b30715f/roles/e962a2a280224f9fbcd0fdc02262993d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:54:17.942103 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-2ded744a-befe-4e6b-a468-1578e7598449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:17.946226 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdbc920-567a-4d31-ac9d-2f3aa6c5840f None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:18.068174 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdbc920-567a-4d31-ac9d-2f3aa6c5840f None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:18.108912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfc3fa-c328-42f4-805f-1ff42d6c69e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:18.109860 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfc3fa-c328-42f4-805f-1ff42d6c69e5 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:18.110198 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfc3fa-c328-42f4-805f-1ff42d6c69e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:18.110988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfc3fa-c328-42f4-805f-1ff42d6c69e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:18.111728 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bfc3fa-c328-42f4-805f-1ff42d6c69e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:18.112040 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bfc3fa-c328-42f4-805f-1ff42d6c69e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:18.113786 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bfc3fa-c328-42f4-805f-1ff42d6c69e5 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:18.124350 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 306/625] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 14 01:54:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:18.125748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:18.130513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfc3fa-c328-42f4-805f-1ff42d6c69e5 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:18.135046 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 320/626] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:54:18.245914 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfc3fa-c328-42f4-805f-1ff42d6c69e5 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:18.282199 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39013c3c-0781-48bf-929f-be1700bf0ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:18.282824 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39013c3c-0781-48bf-929f-be1700bf0ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:18.283099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39013c3c-0781-48bf-929f-be1700bf0ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:18.283871 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39013c3c-0781-48bf-929f-be1700bf0ef6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:18.295296 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39013c3c-0781-48bf-929f-be1700bf0ef6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:18.296802 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39013c3c-0781-48bf-929f-be1700bf0ef6 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:18.311269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-39013c3c-0781-48bf-929f-be1700bf0ef6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:18.311269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:18.311269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:18.311269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:18.311269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:18.311269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:18.311269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:18.311269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:18.311269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:18.311269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:18.311269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:18.311269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:18.311269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:18.311269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:18.311269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:18.311269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:18.312693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:18.312693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:18.312693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:18.312693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:18.312693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:18.312693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:18.312693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:18.312693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:18.312693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:18.312693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:18.312693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:18.312693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:18.312693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:18.312693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:18.312693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:18.312693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:18.312693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:18.313600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:18.313600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:18.313600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:18.313600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 307/627] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:18.318555 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded744a-befe-4e6b-a468-1578e7598449 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:18.336490 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:18.460088 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded744a-befe-4e6b-a468-1578e7598449 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:18.484613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e7f89-e7e4-4a01-b82f-89762f13a05a None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:18.484960 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e7f89-e7e4-4a01-b82f-89762f13a05a None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:18.485142 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e7f89-e7e4-4a01-b82f-89762f13a05a None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:18.486815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 308/628] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:54:18.491980 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0af4d-7095-41a2-b9b6-80fb2991ba89 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:18.492264 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0af4d-7095-41a2-b9b6-80fb2991ba89 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:18.492535 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0af4d-7095-41a2-b9b6-80fb2991ba89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:18.506824 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24a718-b080-4c3b-b957-d214a5162307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:18.507362 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24a718-b080-4c3b-b957-d214a5162307 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:18.507632 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24a718-b080-4c3b-b957-d214a5162307 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:18.507902 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24a718-b080-4c3b-b957-d214a5162307 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:18.508594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24a718-b080-4c3b-b957-d214a5162307 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:18.508894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24a718-b080-4c3b-b957-d214a5162307 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:18.510368 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24a718-b080-4c3b-b957-d214a5162307 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:18.514094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-2df0af4d-7095-41a2-b9b6-80fb2991ba89 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:18.515519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-2df0af4d-7095-41a2-b9b6-80fb2991ba89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:18.526022 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 321/629] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:54:18.526022 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24a718-b080-4c3b-b957-d214a5162307 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:18.573799 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:18.598036 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 309/630] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:54:18.599274 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0af4d-7095-41a2-b9b6-80fb2991ba89 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:18.623155 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:18.651341 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24a718-b080-4c3b-b957-d214a5162307 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:18.689790 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d4a79d-bafb-4d28-b4ac-2546670f5de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:18.690727 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4a79d-bafb-4d28-b4ac-2546670f5de6 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:18.691191 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4a79d-bafb-4d28-b4ac-2546670f5de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:18.691641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4a79d-bafb-4d28-b4ac-2546670f5de6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:18.704763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d4a79d-bafb-4d28-b4ac-2546670f5de6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:18.711914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-84d4a79d-bafb-4d28-b4ac-2546670f5de6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:18.711914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:18.711914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:18.711914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:18.711914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:18.711914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:18.711914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:18.711914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:18.711914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:18.711914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:18.711914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:18.711914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:18.711914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:18.711914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:18.711914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:18.711914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:18.712884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:18.712884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:18.712884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:18.712884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:18.712884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:18.712884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:18.712884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:18.712884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:18.712884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:18.712884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:18.712884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:18.712884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:18.712884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:18.712884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:18.712884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:18.712884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:18.713749 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:18.713749 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:18.713749 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:18.713749 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:18.713749 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:18.713749 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:18.713749 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:18.713749 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:18.713749 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:18.713749 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:18.713749 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:18.713749 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:18.716634 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 322/631] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:54:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:18.723030 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d4a79d-bafb-4d28-b4ac-2546670f5de6 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:18.821282 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0af4d-7095-41a2-b9b6-80fb2991ba89 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:18.882187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe55b7-d0b9-4932-aa6f-3ce6e1db1469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:18.883063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe55b7-d0b9-4932-aa6f-3ce6e1db1469 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:18.883613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe55b7-d0b9-4932-aa6f-3ce6e1db1469 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:18.884995 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe55b7-d0b9-4932-aa6f-3ce6e1db1469 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:18.897840 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe55b7-d0b9-4932-aa6f-3ce6e1db1469 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:18.899613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe55b7-d0b9-4932-aa6f-3ce6e1db1469 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:18.905536 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d4a79d-bafb-4d28-b4ac-2546670f5de6 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:18.918277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-cbbe55b7-d0b9-4932-aa6f-3ce6e1db1469 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:54:18.918277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:18.918277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:18.918277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:18.918277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:18.918277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:18.918277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:18.918277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:18.918277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:18.918277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:18.918277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:18.918277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:18.918277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:18.918277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:18.918277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:18.918277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:18.919495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:18.919495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:18.919495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:18.919495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:18.919495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:18.919495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:18.919495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:18.919495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:18.919495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:18.919495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:18.919495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:18.919495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:18.919495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:18.919495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:18.919495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:18.919495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:18.919495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:18.919495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:18.920525 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:18.920525 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:18.920525 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:18.920525 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:18.920525 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:18.920525 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:18.920525 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:18.920525 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:18.920525 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:54:18.920525 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:18.922647 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 310/632] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Sep 14 01:54:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:18.923708 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe55b7-d0b9-4932-aa6f-3ce6e1db1469 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:18.939653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:18.946017 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b81be-676a-4dc9-8c36-9b5410833821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:18.946716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b81be-676a-4dc9-8c36-9b5410833821 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:18.947225 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b81be-676a-4dc9-8c36-9b5410833821 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:18.947769 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b81be-676a-4dc9-8c36-9b5410833821 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:18.948646 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b81be-676a-4dc9-8c36-9b5410833821 None admin] RBAC: Adding query filter params (name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:18.949255 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b81be-676a-4dc9-8c36-9b5410833821 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:18.950869 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b81be-676a-4dc9-8c36-9b5410833821 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:18.964270 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 323/633] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 14 01:54:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:18.964270 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b81be-676a-4dc9-8c36-9b5410833821 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:18.975049 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:19.056020 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe55b7-d0b9-4932-aa6f-3ce6e1db1469 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:19.070612 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b81be-676a-4dc9-8c36-9b5410833821 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:19.102499 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac4748-8435-49b1-9d9a-536d45eca0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:19.103671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac4748-8435-49b1-9d9a-536d45eca0dc None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:19.104227 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac4748-8435-49b1-9d9a-536d45eca0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:19.104673 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac4748-8435-49b1-9d9a-536d45eca0dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:19.106083 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f421b-19e3-4b8a-8ed8-fc172f4e92b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:19.106685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f421b-19e3-4b8a-8ed8-fc172f4e92b6 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:19.106965 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f421b-19e3-4b8a-8ed8-fc172f4e92b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:19.107231 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f421b-19e3-4b8a-8ed8-fc172f4e92b6 None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:19.107937 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6f421b-19e3-4b8a-8ed8-fc172f4e92b6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:19.108236 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6f421b-19e3-4b8a-8ed8-fc172f4e92b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:19.109648 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6f421b-19e3-4b8a-8ed8-fc172f4e92b6 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:19.118857 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ac4748-8435-49b1-9d9a-536d45eca0dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:19.126401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-e3ac4748-8435-49b1-9d9a-536d45eca0dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:19.126401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:19.126401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:19.126401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:19.126401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:19.126401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:19.126401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:19.126401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:19.126401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:19.126401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:19.126401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:19.126401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:19.126401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:19.126401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:19.126401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:19.126401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:19.127895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:19.127895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:19.127895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:19.127895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:19.127895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:19.127895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:19.127895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:19.127895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:19.127895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:19.127895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:19.127895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:19.127895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:19.127895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:19.127895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:19.127895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:19.127895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:19.128758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:19.128758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:19.128758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:19.128758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:19.128758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:19.128758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:19.128758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:19.128758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:19.128758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:19.128758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:19.128758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:19.128758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:19.129409 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 324/634] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Sep 14 01:54:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:19.129409 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f421b-19e3-4b8a-8ed8-fc172f4e92b6 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:19.136300 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 311/635] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:54:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:19.136300 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac4748-8435-49b1-9d9a-536d45eca0dc None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:19.187522 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:19.243722 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:19.246492 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac4748-8435-49b1-9d9a-536d45eca0dc None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:19.248780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f421b-19e3-4b8a-8ed8-fc172f4e92b6 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:19.281910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10daf63e-627e-48ba-afaa-aa18f45bfe9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:19.282460 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10daf63e-627e-48ba-afaa-aa18f45bfe9a None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:19.282737 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10daf63e-627e-48ba-afaa-aa18f45bfe9a None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:19.283002 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10daf63e-627e-48ba-afaa-aa18f45bfe9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:19.288774 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338471a-2a5a-4746-a404-87e18d7693f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:19.289623 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338471a-2a5a-4746-a404-87e18d7693f8 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:19.290133 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338471a-2a5a-4746-a404-87e18d7693f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:19.290598 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338471a-2a5a-4746-a404-87e18d7693f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:19.301807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338471a-2a5a-4746-a404-87e18d7693f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:19.303893 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338471a-2a5a-4746-a404-87e18d7693f8 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:19.323307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-a338471a-2a5a-4746-a404-87e18d7693f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:19.323307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:19.323307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:19.323307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:19.323307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:19.323307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:19.323307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:19.323307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:19.323307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:19.323307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:19.323307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:19.323307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:19.323307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:19.323307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:19.323307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:19.323307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:19.324391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:19.324391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:19.324391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:19.324391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:19.324391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:19.324391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:19.324391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:19.324391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:19.324391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:19.324391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:19.324391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:19.324391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:19.324391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:19.324391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:19.324391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:19.324391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:19.324391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:19.325355 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:19.325355 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:19.325355 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:19.327712 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 325/636] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:19.328960 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338471a-2a5a-4746-a404-87e18d7693f8 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:19.368838 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10daf63e-627e-48ba-afaa-aa18f45bfe9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:19.371428 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10daf63e-627e-48ba-afaa-aa18f45bfe9a None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:19.459118 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338471a-2a5a-4746-a404-87e18d7693f8 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:19.461507 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:19.487771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:19.495672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 312/637] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:54:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:54:19.495672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10daf63e-627e-48ba-afaa-aa18f45bfe9a None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:19.505374 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ebf7a2-abbe-4415-b817-87f880ed9249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:19.506292 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebf7a2-abbe-4415-b817-87f880ed9249 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:19.507006 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebf7a2-abbe-4415-b817-87f880ed9249 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:19.507276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebf7a2-abbe-4415-b817-87f880ed9249 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:19.508081 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ebf7a2-abbe-4415-b817-87f880ed9249 None admin] RBAC: Adding query filter params (name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:19.508383 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ebf7a2-abbe-4415-b817-87f880ed9249 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:19.509963 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ebf7a2-abbe-4415-b817-87f880ed9249 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:19.523461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 326/638] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 14 01:54:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:19.523461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ee3a5-1b4f-4766-8d9b-a0c681922ac3 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:19.523461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ee3a5-1b4f-4766-8d9b-a0c681922ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:19.523461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ee3a5-1b4f-4766-8d9b-a0c681922ac3 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:19.523461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 327/639] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:19.523461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ee3a5-1b4f-4766-8d9b-a0c681922ac3 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:19.599134 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10daf63e-627e-48ba-afaa-aa18f45bfe9a None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:19.628280 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ee3a5-1b4f-4766-8d9b-a0c681922ac3 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:19.634385 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883ba6a-360c-4627-9eb8-b3d2a4e3a0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:19.634995 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883ba6a-360c-4627-9eb8-b3d2a4e3a0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:19.635270 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883ba6a-360c-4627-9eb8-b3d2a4e3a0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:19.635925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883ba6a-360c-4627-9eb8-b3d2a4e3a0b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:19.636579 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d883ba6a-360c-4627-9eb8-b3d2a4e3a0b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:19.636876 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d883ba6a-360c-4627-9eb8-b3d2a4e3a0b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:19.638617 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d883ba6a-360c-4627-9eb8-b3d2a4e3a0b6 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:19.651665 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 313/640] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:19.652468 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883ba6a-360c-4627-9eb8-b3d2a4e3a0b6 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:19.667776 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe7f30b-10f3-4393-8f94-353bc438ce0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:19.668411 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7f30b-10f3-4393-8f94-353bc438ce0d None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:19.668688 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7f30b-10f3-4393-8f94-353bc438ce0d None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:19.668959 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7f30b-10f3-4393-8f94-353bc438ce0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:19.680537 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe7f30b-10f3-4393-8f94-353bc438ce0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:19.682484 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe7f30b-10f3-4393-8f94-353bc438ce0d None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:19.696766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-6fe7f30b-10f3-4393-8f94-353bc438ce0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:19.696766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:19.696766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:19.696766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:19.696766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:19.696766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:19.696766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:19.696766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:19.696766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:19.696766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:19.696766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:19.696766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:19.696766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:19.696766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:19.696766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:19.696766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:19.697726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:19.697726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:19.697726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:19.697726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:19.697726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:19.697726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:19.697726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:19.697726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:19.697726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:19.697726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:19.697726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:19.697726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:19.697726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:19.697726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:19.697726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:19.697726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:19.697726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:19.698706 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:19.698706 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:19.698706 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:19.698706 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 328/641] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:19.699327 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b395d27-f26f-4413-b1dd-0cde5ce6651b None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:19.699327 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b395d27-f26f-4413-b1dd-0cde5ce6651b None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:19.699498 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b395d27-f26f-4413-b1dd-0cde5ce6651b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:19.717042 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-0b395d27-f26f-4413-b1dd-0cde5ce6651b None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:19.718255 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-0b395d27-f26f-4413-b1dd-0cde5ce6651b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:19.719144 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:19.729682 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:19.763949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883ba6a-360c-4627-9eb8-b3d2a4e3a0b6 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:19.790351 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 329/642] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:54:19.794174 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b395d27-f26f-4413-b1dd-0cde5ce6651b None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:19.817719 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10be51-3a62-4a69-90b9-5ac9548f5e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:19.818870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10be51-3a62-4a69-90b9-5ac9548f5e6a None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:19.819219 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10be51-3a62-4a69-90b9-5ac9548f5e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:19.819561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10be51-3a62-4a69-90b9-5ac9548f5e6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:19.833576 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10be51-3a62-4a69-90b9-5ac9548f5e6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:19.835307 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10be51-3a62-4a69-90b9-5ac9548f5e6a None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:19.850225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-da10be51-3a62-4a69-90b9-5ac9548f5e6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:19.850225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:19.850225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:19.850225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:19.850225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:19.850225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:19.850225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:19.850225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:19.850225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:19.850225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:19.850225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:19.850225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:19.850225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:19.850225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:19.850225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:19.850225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:19.851277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:19.851277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:19.851277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:19.851277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:19.851277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:19.851277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:19.851277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:19.851277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:19.851277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:19.851277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:19.851277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:19.851277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:19.851277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:19.851277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:19.851277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:19.851277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:19.851277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:19.852216 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:19.852216 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:19.852216 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:19.858318 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 314/643] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:19.859340 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10be51-3a62-4a69-90b9-5ac9548f5e6a None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:19.907886 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b395d27-f26f-4413-b1dd-0cde5ce6651b None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:19.954963 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e10b3-4aa3-4197-8bb8-fd66c13bf0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:19.955675 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e10b3-4aa3-4197-8bb8-fd66c13bf0b9 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:19.955953 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e10b3-4aa3-4197-8bb8-fd66c13bf0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:19.956222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e10b3-4aa3-4197-8bb8-fd66c13bf0b9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:19.963277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10be51-3a62-4a69-90b9-5ac9548f5e6a None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:19.966028 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e10b3-4aa3-4197-8bb8-fd66c13bf0b9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:19.967758 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e10b3-4aa3-4197-8bb8-fd66c13bf0b9 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:19.979108 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:19.988969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-b74e10b3-4aa3-4197-8bb8-fd66c13bf0b9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 14 01:54:19.988969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:19.988969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:19.988969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:19.988969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:19.988969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:19.988969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:19.988969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:19.988969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:19.988969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:19.988969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:19.988969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:19.988969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:19.988969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:19.988969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:19.988969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:19.989942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:19.989942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:19.989942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:19.989942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:19.989942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:19.989942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:19.989942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:19.989942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:19.989942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:19.989942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:19.989942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:19.989942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:19.989942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:19.989942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:19.989942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:19.989942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:19.989942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:19.989942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:19.991006 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:19.991006 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:19.991006 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:19.991006 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:19.991006 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:19.991006 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:19.991006 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:19.991006 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:19.991006 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 14 01:54:19.991006 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:19.991656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 330/644] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:54:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:19.994770 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e10b3-4aa3-4197-8bb8-fd66c13bf0b9 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:20.009036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:20.013275 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f82c9d-58c2-4cbb-9506-9a27971fd88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:20.014638 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f82c9d-58c2-4cbb-9506-9a27971fd88b None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:20.015119 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f82c9d-58c2-4cbb-9506-9a27971fd88b None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:20.016341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f82c9d-58c2-4cbb-9506-9a27971fd88b None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:20.017681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f82c9d-58c2-4cbb-9506-9a27971fd88b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:20.018553 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f82c9d-58c2-4cbb-9506-9a27971fd88b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:20.020801 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f82c9d-58c2-4cbb-9506-9a27971fd88b None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:20.038906 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 315/645] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:20.040041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f82c9d-58c2-4cbb-9506-9a27971fd88b None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:20.110049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e10b3-4aa3-4197-8bb8-fd66c13bf0b9 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:20.145377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f82c9d-58c2-4cbb-9506-9a27971fd88b None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:20.148226 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d32c9-2ab0-47df-8505-212aa4741bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:20.148359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d32c9-2ab0-47df-8505-212aa4741bad None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:20.148596 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d32c9-2ab0-47df-8505-212aa4741bad None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:20.148853 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d32c9-2ab0-47df-8505-212aa4741bad None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:20.149477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d32c9-2ab0-47df-8505-212aa4741bad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:20.149769 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d32c9-2ab0-47df-8505-212aa4741bad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:20.151182 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d32c9-2ab0-47df-8505-212aa4741bad None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:20.168506 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 331/646] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:20.168506 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d32c9-2ab0-47df-8505-212aa4741bad None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:20.181100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef68379-d09a-4646-a859-1ee5dfc96efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:20.181705 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef68379-d09a-4646-a859-1ee5dfc96efb None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:20.182003 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef68379-d09a-4646-a859-1ee5dfc96efb None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:20.182279 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef68379-d09a-4646-a859-1ee5dfc96efb None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:20.182993 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef68379-d09a-4646-a859-1ee5dfc96efb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:20.183291 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef68379-d09a-4646-a859-1ee5dfc96efb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:20.184717 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef68379-d09a-4646-a859-1ee5dfc96efb None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:20.215559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 316/647] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Sep 14 01:54:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:20.215559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef68379-d09a-4646-a859-1ee5dfc96efb None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:20.243060 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:20.250252 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:20.281570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d32c9-2ab0-47df-8505-212aa4741bad None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:20.316665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b195164-7052-4344-8646-2c071da30fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:20.317292 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b195164-7052-4344-8646-2c071da30fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:20.317580 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b195164-7052-4344-8646-2c071da30fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:20.317854 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b195164-7052-4344-8646-2c071da30fa5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:20.321019 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef68379-d09a-4646-a859-1ee5dfc96efb None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:20.324837 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b195164-7052-4344-8646-2c071da30fa5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:20.326328 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b195164-7052-4344-8646-2c071da30fa5 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:20.337755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-5b195164-7052-4344-8646-2c071da30fa5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:20.337755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:20.337755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:20.337755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:20.337755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:20.337755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:20.337755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:20.337755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:20.337755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:20.337755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:20.337755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:20.337755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:20.337755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:20.337755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:20.337755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:20.338977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:20.338977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:20.338977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:20.338977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:20.338977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:20.338977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:20.338977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:20.338977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:20.338977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:20.338977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:20.338977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:20.338977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:20.338977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:20.338977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:20.338977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:20.338977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:20.338977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:20.339886 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:20.339886 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:20.339886 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:20.339886 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:20.339886 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:20.339886 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:20.339886 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:20.339886 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:20.339886 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 332/648] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:20.342836 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b195164-7052-4344-8646-2c071da30fa5 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:20.358740 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ba0dc-4904-4920-baa8-69a99e629822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:20.359369 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ba0dc-4904-4920-baa8-69a99e629822 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:20.359643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ba0dc-4904-4920-baa8-69a99e629822 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:20.359915 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ba0dc-4904-4920-baa8-69a99e629822 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:20.370688 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738ba0dc-4904-4920-baa8-69a99e629822 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:20.375430 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-738ba0dc-4904-4920-baa8-69a99e629822 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:20.375430 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:20.375430 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:20.375430 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:20.375430 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:20.375430 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:20.375430 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:20.375430 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:20.375430 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:20.375430 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:20.375430 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:20.375430 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:20.375430 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:20.375430 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:20.375430 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:20.375430 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:20.376372 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:20.376372 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:20.376372 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:20.376372 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:20.376372 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:20.376372 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:20.376372 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:20.376372 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:20.376372 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:20.376372 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:20.376372 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:20.376372 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:20.376372 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:20.376372 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:20.376372 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:20.376372 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:20.377324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:20.377324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:20.377324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:20.377324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:20.377324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:20.377324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:20.377324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:20.377324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:20.377324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:20.377324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:20.377324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:20.377324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:20.377324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 317/649] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:54:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:20.377324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ba0dc-4904-4920-baa8-69a99e629822 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:20.463353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b195164-7052-4344-8646-2c071da30fa5 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:20.477886 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:20.488297 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ba0dc-4904-4920-baa8-69a99e629822 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:20.495935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:20.506457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00bb7fb-ec6f-4ea6-942f-3021fe55c826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:20.507090 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bb7fb-ec6f-4ea6-942f-3021fe55c826 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:20.507405 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bb7fb-ec6f-4ea6-942f-3021fe55c826 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:20.507636 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bb7fb-ec6f-4ea6-942f-3021fe55c826 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:20.515639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00bb7fb-ec6f-4ea6-942f-3021fe55c826 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:20.517085 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00bb7fb-ec6f-4ea6-942f-3021fe55c826 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:20.524970 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d490f8b-283c-49b6-babd-0741a1f66f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:20.525578 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d490f8b-283c-49b6-babd-0741a1f66f62 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:20.525858 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d490f8b-283c-49b6-babd-0741a1f66f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:20.526187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d490f8b-283c-49b6-babd-0741a1f66f62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:20.528367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-f00bb7fb-ec6f-4ea6-942f-3021fe55c826 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 01:54:20.528367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:20.528367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:20.528367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:20.528367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:20.528367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:20.528367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:20.528367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:20.528367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:20.528367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:20.528367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:20.528367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:20.528367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:20.528367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:20.528367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:20.528367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:20.529340 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:20.529340 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:20.529340 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:20.529340 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:20.529340 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:20.529340 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:20.529340 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:20.529340 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:20.529340 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:20.529340 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:20.529340 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:20.529340 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:20.529340 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:20.529340 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:20.529340 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:20.529340 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:20.529340 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:20.529340 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:20.530338 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:20.530338 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:20.530338 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:20.530338 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:20.530338 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:20.530338 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:20.530338 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:20.530338 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:20.530338 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 01:54:20.530338 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:20.530338 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 333/650] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:54:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:20.530338 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00bb7fb-ec6f-4ea6-942f-3021fe55c826 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:20.599434 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d490f8b-283c-49b6-babd-0741a1f66f62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:20.600910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d490f8b-283c-49b6-babd-0741a1f66f62 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:20.634018 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00bb7fb-ec6f-4ea6-942f-3021fe55c826 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:20.666479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a23dc-448d-4407-b610-4878cfc8309c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:20.667074 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a23dc-448d-4407-b610-4878cfc8309c None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:20.667353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a23dc-448d-4407-b610-4878cfc8309c None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:20.667622 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a23dc-448d-4407-b610-4878cfc8309c None admin] PATH_INFO: `/v3/projects` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:20.668290 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a23dc-448d-4407-b610-4878cfc8309c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:20.668586 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a23dc-448d-4407-b610-4878cfc8309c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:20.670245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a23dc-448d-4407-b610-4878cfc8309c None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:20.683154 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 334/651] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:20.683154 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a23dc-448d-4407-b610-4878cfc8309c None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:20.692664 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:20.703106 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 318/652] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:54:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:54:20.704355 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f760c-d21f-4797-a523-783ea340eb5a None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:20.704597 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f760c-d21f-4797-a523-783ea340eb5a None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:20.704860 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f760c-d21f-4797-a523-783ea340eb5a None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:20.706266 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 319/653] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:20.706836 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f760c-d21f-4797-a523-783ea340eb5a None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:20.739676 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:20.793000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a23dc-448d-4407-b610-4878cfc8309c None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:20.807966 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f760c-d21f-4797-a523-783ea340eb5a None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:20.826968 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4cab3f-3790-4bb7-b62f-e93a896ad895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:20.827568 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4cab3f-3790-4bb7-b62f-e93a896ad895 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:20.827841 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4cab3f-3790-4bb7-b62f-e93a896ad895 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:20.828121 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4cab3f-3790-4bb7-b62f-e93a896ad895 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:20.828748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4cab3f-3790-4bb7-b62f-e93a896ad895 None admin] RBAC: Adding query filter params (name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:20.829043 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4cab3f-3790-4bb7-b62f-e93a896ad895 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:20.831024 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4cab3f-3790-4bb7-b62f-e93a896ad895 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:20.839152 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 335/654] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 14 01:54:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:20.839927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4cab3f-3790-4bb7-b62f-e93a896ad895 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:20.851406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d9bd4-cd68-4081-998b-d3194b7df0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:20.852016 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d9bd4-cd68-4081-998b-d3194b7df0cb None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:20.852663 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d9bd4-cd68-4081-998b-d3194b7df0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:20.852969 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d9bd4-cd68-4081-998b-d3194b7df0cb None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:20.853701 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d9bd4-cd68-4081-998b-d3194b7df0cb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:20.854024 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d9bd4-cd68-4081-998b-d3194b7df0cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:20.855496 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d9bd4-cd68-4081-998b-d3194b7df0cb None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:20.872264 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 320/655] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Sep 14 01:54:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:20.873053 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d9bd4-cd68-4081-998b-d3194b7df0cb None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:20.924470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:20.940549 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4cab3f-3790-4bb7-b62f-e93a896ad895 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:20.949063 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:20.973463 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9546d12-fc7f-4d3e-be56-eae2f7d464ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:20.973797 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d9bd4-cd68-4081-998b-d3194b7df0cb None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:20.974088 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9546d12-fc7f-4d3e-be56-eae2f7d464ff None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:20.974366 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9546d12-fc7f-4d3e-be56-eae2f7d464ff None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:20.974633 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9546d12-fc7f-4d3e-be56-eae2f7d464ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:20.975333 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9546d12-fc7f-4d3e-be56-eae2f7d464ff None admin] RBAC: Adding query filter params (role.id=e962a2a280224f9fbcd0fdc02262993d, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=f721a3a4eca941989685448a87bb9a19) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:20.975632 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9546d12-fc7f-4d3e-be56-eae2f7d464ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:20.977111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9546d12-fc7f-4d3e-be56-eae2f7d464ff None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:20.982722 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 336/656] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:54:20 2022] GET /identity/v3/role_assignments?user.id=f721a3a4eca941989685448a87bb9a19&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=e962a2a280224f9fbcd0fdc02262993d => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:20.983917 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf33e0-1546-49cd-908d-f1480abc0dd1 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:20.984199 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf33e0-1546-49cd-908d-f1480abc0dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:20.984466 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf33e0-1546-49cd-908d-f1480abc0dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:21.004187 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-89bf33e0-1546-49cd-908d-f1480abc0dd1 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:21.005329 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-89bf33e0-1546-49cd-908d-f1480abc0dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:21.015224 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50fff6-b4a9-4b2c-ae07-be794223b835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:21.016330 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50fff6-b4a9-4b2c-ae07-be794223b835 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:21.016607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50fff6-b4a9-4b2c-ae07-be794223b835 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:21.016874 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50fff6-b4a9-4b2c-ae07-be794223b835 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:21.028616 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50fff6-b4a9-4b2c-ae07-be794223b835 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:21.030095 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50fff6-b4a9-4b2c-ae07-be794223b835 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:21.046655 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-6c50fff6-b4a9-4b2c-ae07-be794223b835 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:21.046655 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:21.046655 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:21.046655 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:21.046655 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:21.046655 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:21.046655 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:21.046655 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:21.046655 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:21.046655 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:21.046655 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:21.046655 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:21.046655 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:21.046655 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:21.046655 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:21.046655 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:21.047746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:21.047746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:21.047746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:21.047746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:21.047746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:21.047746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:21.047746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:21.047746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:21.047746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:21.047746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:21.047746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:21.047746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:21.047746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:21.047746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:21.047746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:21.047746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:21.047746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:21.054239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:21.054239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:21.054239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:21.054239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 321/657] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:21.054239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50fff6-b4a9-4b2c-ae07-be794223b835 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:21.086355 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 337/658] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:54:21.095051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf33e0-1546-49cd-908d-f1480abc0dd1 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:21.162455 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:21.165638 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50fff6-b4a9-4b2c-ae07-be794223b835 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:21.207187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b455da-a13e-48af-afad-c7b153a3193e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:21.207835 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b455da-a13e-48af-afad-c7b153a3193e None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:21.208112 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b455da-a13e-48af-afad-c7b153a3193e None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:21.208379 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b455da-a13e-48af-afad-c7b153a3193e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:21.215919 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf33e0-1546-49cd-908d-f1480abc0dd1 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:21.217459 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:21.222047 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b455da-a13e-48af-afad-c7b153a3193e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:21.223725 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b455da-a13e-48af-afad-c7b153a3193e None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:21.246862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-12b455da-a13e-48af-afad-c7b153a3193e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:21.246862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:21.246862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:21.246862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:21.246862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:21.246862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:21.246862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:21.246862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:21.246862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:21.246862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:21.246862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:21.246862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:21.246862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:21.246862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:21.246862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:21.246862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:21.247912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:21.247912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:21.247912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:21.247912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:21.247912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:21.247912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:21.247912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:21.247912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:21.247912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:21.247912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:21.247912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:21.247912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:21.247912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:21.247912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:21.247912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:21.247912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:21.247912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:21.248845 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:21.248845 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:21.248845 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:21.249302 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 322/659] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:21.250767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b455da-a13e-48af-afad-c7b153a3193e None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:21.266010 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a5ab2-05b6-4964-a16f-458657d76a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:21.266694 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a5ab2-05b6-4964-a16f-458657d76a7f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:21.267183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a5ab2-05b6-4964-a16f-458657d76a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:21.267581 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a5ab2-05b6-4964-a16f-458657d76a7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:21.362817 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b455da-a13e-48af-afad-c7b153a3193e None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:21.363827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a5ab2-05b6-4964-a16f-458657d76a7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:21.365291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a5ab2-05b6-4964-a16f-458657d76a7f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:21.411332 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca51510-ca55-4ada-9f27-da45056a1347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:21.411950 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca51510-ca55-4ada-9f27-da45056a1347 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:21.412244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca51510-ca55-4ada-9f27-da45056a1347 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:21.412500 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca51510-ca55-4ada-9f27-da45056a1347 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:21.413137 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca51510-ca55-4ada-9f27-da45056a1347 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:21.413450 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca51510-ca55-4ada-9f27-da45056a1347 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:21.414922 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca51510-ca55-4ada-9f27-da45056a1347 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:21.427957 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 323/660] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:21.428592 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca51510-ca55-4ada-9f27-da45056a1347 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:21.444290 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:21.478079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 338/661] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:54:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:54:21.478079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a5ab2-05b6-4964-a16f-458657d76a7f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:21.478079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:21.544812 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca51510-ca55-4ada-9f27-da45056a1347 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:21.583677 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474280dc-6ff8-4694-b308-0a8d3b02bc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:21.584409 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474280dc-6ff8-4694-b308-0a8d3b02bc2a None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:21.584621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474280dc-6ff8-4694-b308-0a8d3b02bc2a None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:21.584818 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474280dc-6ff8-4694-b308-0a8d3b02bc2a None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:21.585468 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474280dc-6ff8-4694-b308-0a8d3b02bc2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:21.586157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474280dc-6ff8-4694-b308-0a8d3b02bc2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:21.587164 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474280dc-6ff8-4694-b308-0a8d3b02bc2a None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:21.588913 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a5ab2-05b6-4964-a16f-458657d76a7f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:21.599416 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 324/662] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:21.600268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474280dc-6ff8-4694-b308-0a8d3b02bc2a None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:21.631713 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155ccb7-faf8-452f-a5cf-324e99b1be38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:21.632355 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155ccb7-faf8-452f-a5cf-324e99b1be38 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:21.632623 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155ccb7-faf8-452f-a5cf-324e99b1be38 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:21.632901 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155ccb7-faf8-452f-a5cf-324e99b1be38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:21.643068 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155ccb7-faf8-452f-a5cf-324e99b1be38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:21.648000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-6155ccb7-faf8-452f-a5cf-324e99b1be38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:21.648000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:21.648000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:21.648000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:21.648000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:21.648000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:21.648000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:21.648000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:21.648000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:21.648000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:21.648000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:21.648000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:21.648000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:21.648000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:21.648000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:21.648000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:21.648952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:21.648952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:21.648952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:21.648952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:21.648952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:21.648952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:21.648952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:21.648952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:21.648952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:21.648952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:21.648952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:21.648952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:21.648952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:21.648952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:21.648952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:21.648952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:21.649836 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:21.649836 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:21.649836 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:21.649836 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:21.649836 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:21.649836 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:21.649836 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:21.649836 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:21.649836 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:21.649836 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:21.649836 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:21.649836 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:21.649836 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 339/663] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:54:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:21.649836 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155ccb7-faf8-452f-a5cf-324e99b1be38 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:21.688679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:21.713359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:21.732701 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474280dc-6ff8-4694-b308-0a8d3b02bc2a None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:21.762788 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155ccb7-faf8-452f-a5cf-324e99b1be38 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:21.769907 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f15a4-c50a-43e4-badc-6daee5aa5a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:21.770558 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f15a4-c50a-43e4-badc-6daee5aa5a07 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:21.770835 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f15a4-c50a-43e4-badc-6daee5aa5a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:21.771102 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f15a4-c50a-43e4-badc-6daee5aa5a07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:21.778034 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f15a4-c50a-43e4-badc-6daee5aa5a07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:21.779477 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f15a4-c50a-43e4-badc-6daee5aa5a07 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:21.792004 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-df6f15a4-c50a-43e4-badc-6daee5aa5a07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:21.792004 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:21.792004 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:21.792004 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:21.792004 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:21.792004 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:21.792004 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:21.792004 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:21.792004 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:21.792004 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:21.792004 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:21.792004 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:21.792004 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:21.792004 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:21.792004 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:21.793292 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:21.793292 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:21.793292 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:21.793292 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:21.793292 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:21.793292 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:21.793292 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:21.793292 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:21.793292 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:21.793292 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:21.793292 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:21.793292 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:21.793292 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:21.793292 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:21.793292 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:21.793292 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:21.793292 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:21.794887 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:21.794887 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:21.794887 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:21.794887 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:21.794887 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:21.794887 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:21.794887 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:21.794887 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:21.795748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 325/664] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:21.795748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f15a4-c50a-43e4-badc-6daee5aa5a07 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:21.809207 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20185145-164a-43ef-8236-c594b6d19f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:21.809836 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20185145-164a-43ef-8236-c594b6d19f96 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:21.810282 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20185145-164a-43ef-8236-c594b6d19f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:21.810558 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20185145-164a-43ef-8236-c594b6d19f96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:21.822138 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20185145-164a-43ef-8236-c594b6d19f96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:21.823629 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20185145-164a-43ef-8236-c594b6d19f96 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:21.839530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-20185145-164a-43ef-8236-c594b6d19f96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:21.839530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:21.839530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:21.839530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:21.839530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:21.839530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:21.839530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:21.839530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:21.839530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:21.839530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:21.839530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:21.839530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:21.839530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:21.839530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:21.839530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:21.839530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:21.840553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:21.840553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:21.840553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:21.840553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:21.840553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:21.840553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:21.840553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:21.840553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:21.840553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:21.840553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:21.840553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:21.840553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:21.840553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:21.840553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:21.840553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:21.840553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:21.840553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:21.841472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:21.841472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:21.841472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:21.841472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 340/665] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:21.841472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20185145-164a-43ef-8236-c594b6d19f96 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:21.913932 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f15a4-c50a-43e4-badc-6daee5aa5a07 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:21.946362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:21.952594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc4127-d3c8-4668-9aa6-d497efdc9d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:21.953233 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc4127-d3c8-4668-9aa6-d497efdc9d23 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:21.953358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20185145-164a-43ef-8236-c594b6d19f96 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:21.953537 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc4127-d3c8-4668-9aa6-d497efdc9d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:21.953780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc4127-d3c8-4668-9aa6-d497efdc9d23 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:21.962268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc4127-d3c8-4668-9aa6-d497efdc9d23 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:21.964422 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc4127-d3c8-4668-9aa6-d497efdc9d23 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:21.968346 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:21.978527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-a1fc4127-d3c8-4668-9aa6-d497efdc9d23 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 01:54:21.978527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:21.978527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:21.978527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:21.978527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:21.978527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:21.978527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:21.978527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:21.978527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:21.978527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:21.978527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:21.978527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:21.978527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:21.978527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:21.978527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:21.978527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:21.979527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:21.979527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:21.979527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:21.979527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:21.979527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:21.979527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:21.979527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:21.979527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:21.979527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:21.979527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:21.979527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:21.979527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:21.979527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:21.979527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:21.979527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:21.979527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:21.979527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:21.979527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:21.980516 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:21.980516 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:21.980516 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:21.980516 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:21.980516 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:21.980516 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:21.980516 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:21.980516 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:21.980516 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 01:54:21.980516 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:21.980516 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 326/666] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:54:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:21.981146 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc4127-d3c8-4668-9aa6-d497efdc9d23 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:21.996996 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5b178-9e81-4965-a308-26ba84101748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:21.997612 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5b178-9e81-4965-a308-26ba84101748 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:21.997902 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5b178-9e81-4965-a308-26ba84101748 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:21.998195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5b178-9e81-4965-a308-26ba84101748 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:21.998841 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5b178-9e81-4965-a308-26ba84101748 None admin] RBAC: Adding query filter params (name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:21.999178 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5b178-9e81-4965-a308-26ba84101748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:22.000528 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5b178-9e81-4965-a308-26ba84101748 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:22.010803 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 341/667] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 14 01:54:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:22.016820 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5b178-9e81-4965-a308-26ba84101748 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:22.095521 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc4127-d3c8-4668-9aa6-d497efdc9d23 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:22.128253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5b178-9e81-4965-a308-26ba84101748 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:22.132116 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5708f64-cc33-4b1d-8022-23d0cd717245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:22.132715 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5708f64-cc33-4b1d-8022-23d0cd717245 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:22.132988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5708f64-cc33-4b1d-8022-23d0cd717245 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:22.133377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5708f64-cc33-4b1d-8022-23d0cd717245 None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:22.134088 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5708f64-cc33-4b1d-8022-23d0cd717245 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:22.134485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5708f64-cc33-4b1d-8022-23d0cd717245 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:22.135980 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5708f64-cc33-4b1d-8022-23d0cd717245 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:22.148324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 327/668] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:22.149127 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5708f64-cc33-4b1d-8022-23d0cd717245 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:22.171695 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d48d5-53ca-43a1-9f8e-d87ca0d7cb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:22.172924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d48d5-53ca-43a1-9f8e-d87ca0d7cb00 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:22.174628 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d48d5-53ca-43a1-9f8e-d87ca0d7cb00 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:22.175739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d48d5-53ca-43a1-9f8e-d87ca0d7cb00 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:22.177325 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d48d5-53ca-43a1-9f8e-d87ca0d7cb00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:22.177825 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d48d5-53ca-43a1-9f8e-d87ca0d7cb00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:22.181149 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d48d5-53ca-43a1-9f8e-d87ca0d7cb00 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:22.198797 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 342/669] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:22.198797 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d48d5-53ca-43a1-9f8e-d87ca0d7cb00 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:22.211717 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:22.235707 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:22.266960 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5708f64-cc33-4b1d-8022-23d0cd717245 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:22.305364 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e76c5-3c42-41e5-8dec-6fdca71f486b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:22.306100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e76c5-3c42-41e5-8dec-6fdca71f486b None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:22.306451 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e76c5-3c42-41e5-8dec-6fdca71f486b None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:22.306794 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e76c5-3c42-41e5-8dec-6fdca71f486b None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:22.307589 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e76c5-3c42-41e5-8dec-6fdca71f486b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:22.307962 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e76c5-3c42-41e5-8dec-6fdca71f486b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:22.309453 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e76c5-3c42-41e5-8dec-6fdca71f486b None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:22.311239 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d48d5-53ca-43a1-9f8e-d87ca0d7cb00 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:22.333901 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 328/670] 10.222.0.15 () {56 vars in 1139 bytes} [Wed Sep 14 01:54:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:22.333901 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e76c5-3c42-41e5-8dec-6fdca71f486b None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:22.356221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68f8ca-a854-444f-958a-06ec66b84083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:22.356787 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68f8ca-a854-444f-958a-06ec66b84083 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:22.357224 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68f8ca-a854-444f-958a-06ec66b84083 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:22.357498 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68f8ca-a854-444f-958a-06ec66b84083 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:22.369422 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68f8ca-a854-444f-958a-06ec66b84083 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:22.371042 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68f8ca-a854-444f-958a-06ec66b84083 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:22.393139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-0f68f8ca-a854-444f-958a-06ec66b84083 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:22.393139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:22.393139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:22.393139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:22.393139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:22.393139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:22.393139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:22.393139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:22.393139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:22.393139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:22.393139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:22.393139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:22.393139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:22.393139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:22.393139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:22.393139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:22.394210 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:22.394210 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:22.394210 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:22.394210 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:22.394210 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:22.394210 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:22.394210 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:22.394210 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:22.394210 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:22.394210 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:22.394210 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:22.394210 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:22.394210 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:22.394210 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:22.394210 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:22.394210 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:22.394210 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:22.395313 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:22.395313 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:22.395313 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:22.395313 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 343/671] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:22.395313 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68f8ca-a854-444f-958a-06ec66b84083 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:22.447985 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e76c5-3c42-41e5-8dec-6fdca71f486b None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:22.464286 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:22.491148 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5fe00-6bfa-4d45-af1e-6a8ed3dde11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:22.491893 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5fe00-6bfa-4d45-af1e-6a8ed3dde11c None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:22.492261 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5fe00-6bfa-4d45-af1e-6a8ed3dde11c None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:22.492644 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5fe00-6bfa-4d45-af1e-6a8ed3dde11c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:22.493459 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5fe00-6bfa-4d45-af1e-6a8ed3dde11c None admin] RBAC: Adding query filter params (role.id=e962a2a280224f9fbcd0fdc02262993d, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=5a5191afa5f246269d5b4a60972332a6) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:22.493859 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5fe00-6bfa-4d45-af1e-6a8ed3dde11c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:22.495397 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5fe00-6bfa-4d45-af1e-6a8ed3dde11c None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:22.495877 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:22.502796 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 329/672] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:54:22 2022] GET /identity/v3/role_assignments?user.id=5a5191afa5f246269d5b4a60972332a6&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=e962a2a280224f9fbcd0fdc02262993d => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:22.503857 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5fe00-6bfa-4d45-af1e-6a8ed3dde11c None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:22.517744 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68f8ca-a854-444f-958a-06ec66b84083 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:22.560284 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd7ab1-f91a-46af-916a-f90b801d8e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:22.561117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd7ab1-f91a-46af-916a-f90b801d8e8d None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:22.561488 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd7ab1-f91a-46af-916a-f90b801d8e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:22.561854 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd7ab1-f91a-46af-916a-f90b801d8e8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:22.572299 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd7ab1-f91a-46af-916a-f90b801d8e8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:22.574190 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd7ab1-f91a-46af-916a-f90b801d8e8d None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:22.587512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-a5dd7ab1-f91a-46af-916a-f90b801d8e8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:22.587512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:22.587512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:22.587512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:22.587512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:22.587512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:22.587512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:22.587512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:22.587512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:22.587512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:22.587512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:22.587512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:22.587512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:22.587512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:22.587512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:22.588669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:22.588669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:22.588669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:22.588669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:22.588669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:22.588669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:22.588669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:22.588669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:22.588669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:22.588669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:22.588669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:22.588669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:22.588669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:22.588669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:22.588669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:22.588669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:22.588669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:22.589764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:22.589764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:22.589764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:22.589764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:22.589764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:22.589764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:22.589764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:22.589764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:22.595879 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 344/673] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:22.595879 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd7ab1-f91a-46af-916a-f90b801d8e8d None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:22.659518 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5fe00-6bfa-4d45-af1e-6a8ed3dde11c None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:22.707591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb63a6b-8b7b-405c-8607-f746f27e88ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:22.708598 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb63a6b-8b7b-405c-8607-f746f27e88ce None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:22.708882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb63a6b-8b7b-405c-8607-f746f27e88ce None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:22.709149 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb63a6b-8b7b-405c-8607-f746f27e88ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:22.727631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd7ab1-f91a-46af-916a-f90b801d8e8d None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:22.767496 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:22.770573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c683b2e-bbfd-4ae0-b017-6999ee04fd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:22.771577 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c683b2e-bbfd-4ae0-b017-6999ee04fd53 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:22.771859 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c683b2e-bbfd-4ae0-b017-6999ee04fd53 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:22.772323 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c683b2e-bbfd-4ae0-b017-6999ee04fd53 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:22.772904 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c683b2e-bbfd-4ae0-b017-6999ee04fd53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:22.773202 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c683b2e-bbfd-4ae0-b017-6999ee04fd53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:22.774884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c683b2e-bbfd-4ae0-b017-6999ee04fd53 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:22.790506 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 345/674] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:22.791766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c683b2e-bbfd-4ae0-b017-6999ee04fd53 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:22.803735 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:22.806781 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb63a6b-8b7b-405c-8607-f746f27e88ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:22.808310 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb63a6b-8b7b-405c-8607-f746f27e88ce None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:22.904920 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c683b2e-bbfd-4ae0-b017-6999ee04fd53 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:22.945172 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b917e-034f-4b6a-a973-b3246599c982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:22.946037 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b917e-034f-4b6a-a973-b3246599c982 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:22.946478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b917e-034f-4b6a-a973-b3246599c982 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:22.946892 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b917e-034f-4b6a-a973-b3246599c982 None admin] PATH_INFO: `/v3/projects` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:22.947726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b917e-034f-4b6a-a973-b3246599c982 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:22.948172 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b917e-034f-4b6a-a973-b3246599c982 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:22.949733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b917e-034f-4b6a-a973-b3246599c982 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:22.950394 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 330/675] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:54:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:54:22.951333 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb63a6b-8b7b-405c-8607-f746f27e88ce None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:22.967974 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 346/676] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:22.969097 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b917e-034f-4b6a-a973-b3246599c982 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:23.043898 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:23.054603 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:23.084883 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b917e-034f-4b6a-a973-b3246599c982 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:23.089121 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb63a6b-8b7b-405c-8607-f746f27e88ce None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:23.124666 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5cf3d-5b4f-4ca9-b47e-160b831df670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:23.125581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5cf3d-5b4f-4ca9-b47e-160b831df670 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:23.126034 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5cf3d-5b4f-4ca9-b47e-160b831df670 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:23.126447 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5cf3d-5b4f-4ca9-b47e-160b831df670 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:23.131409 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ddcf0-b0ce-4f12-a26f-327bd9b664ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:23.132041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ddcf0-b0ce-4f12-a26f-327bd9b664ce None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:23.132387 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ddcf0-b0ce-4f12-a26f-327bd9b664ce None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:23.132669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ddcf0-b0ce-4f12-a26f-327bd9b664ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:23.135575 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5cf3d-5b4f-4ca9-b47e-160b831df670 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:23.137167 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5cf3d-5b4f-4ca9-b47e-160b831df670 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:23.146212 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ddcf0-b0ce-4f12-a26f-327bd9b664ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:23.147660 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ddcf0-b0ce-4f12-a26f-327bd9b664ce None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:23.150645 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-45b5cf3d-5b4f-4ca9-b47e-160b831df670 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:54:23.150645 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:23.150645 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:23.150645 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:23.150645 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:23.150645 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:23.150645 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:23.150645 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:23.150645 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:23.150645 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:23.150645 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:23.150645 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:23.150645 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:23.150645 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:23.150645 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:23.150645 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:23.151771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:23.151771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:23.151771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:23.151771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:23.151771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:23.151771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:23.151771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:23.151771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:23.151771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:23.151771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:23.151771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:23.151771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:23.151771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:23.151771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:23.151771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:23.151771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:23.151771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:23.151771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:23.152834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:23.152834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:23.152834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:23.152834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:23.152834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:23.152834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:23.152834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:23.152834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:23.152834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:54:23.152834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:23.155184 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 331/677] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:54:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:23.156280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5cf3d-5b4f-4ca9-b47e-160b831df670 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:23.164763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-217ddcf0-b0ce-4f12-a26f-327bd9b664ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:23.164763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:23.164763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:23.164763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:23.164763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:23.164763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:23.164763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:23.164763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:23.164763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:23.164763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:23.164763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:23.164763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:23.164763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:23.164763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:23.164763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:23.164763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:23.165743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:23.165743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:23.165743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:23.165743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:23.165743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:23.165743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:23.165743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:23.165743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:23.165743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:23.165743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:23.165743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:23.165743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:23.165743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:23.165743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:23.165743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:23.165743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:23.165743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:23.167266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:23.167266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:23.167266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:23.167266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 347/678] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:23.167266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ddcf0-b0ce-4f12-a26f-327bd9b664ce None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:23.270037 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5cf3d-5b4f-4ca9-b47e-160b831df670 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:23.310450 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ddcf0-b0ce-4f12-a26f-327bd9b664ce None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:23.316126 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df13575-72eb-4906-8f51-41bd7a398bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:23.316751 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df13575-72eb-4906-8f51-41bd7a398bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:23.317032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df13575-72eb-4906-8f51-41bd7a398bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:23.317306 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df13575-72eb-4906-8f51-41bd7a398bc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:23.318057 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df13575-72eb-4906-8f51-41bd7a398bc3 None admin] RBAC: Adding query filter params (role.id=e962a2a280224f9fbcd0fdc02262993d, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=c6d70ad7054e40c48f8751a3d4fed787) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:23.318367 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df13575-72eb-4906-8f51-41bd7a398bc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:23.319807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df13575-72eb-4906-8f51-41bd7a398bc3 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:23.322233 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:23.329472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 332/679] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:54:23 2022] GET /identity/v3/role_assignments?user.id=c6d70ad7054e40c48f8751a3d4fed787&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=e962a2a280224f9fbcd0fdc02262993d => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:23.331244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df13575-72eb-4906-8f51-41bd7a398bc3 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:23.348527 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:23.373184 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca9613-342d-4f74-962b-7d2c834d399b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:23.374309 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9613-342d-4f74-962b-7d2c834d399b None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:23.374810 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9613-342d-4f74-962b-7d2c834d399b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:23.375327 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9613-342d-4f74-962b-7d2c834d399b None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:23.376315 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca9613-342d-4f74-962b-7d2c834d399b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:23.376936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca9613-342d-4f74-962b-7d2c834d399b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:23.378833 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca9613-342d-4f74-962b-7d2c834d399b None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:23.409007 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 348/680] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Sep 14 01:54:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:23.410802 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d8ba3-9f02-42a7-bead-43ae0df2ce41 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:23.411092 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d8ba3-9f02-42a7-bead-43ae0df2ce41 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:23.411359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d8ba3-9f02-42a7-bead-43ae0df2ce41 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:23.413670 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 349/681] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:23.414828 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d8ba3-9f02-42a7-bead-43ae0df2ce41 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:23.463822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df13575-72eb-4906-8f51-41bd7a398bc3 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:23.507414 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e96e0b-62f7-4893-867c-e41c0eca8203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:23.508040 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e96e0b-62f7-4893-867c-e41c0eca8203 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:23.508356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e96e0b-62f7-4893-867c-e41c0eca8203 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:23.508621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e96e0b-62f7-4893-867c-e41c0eca8203 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:23.509362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e96e0b-62f7-4893-867c-e41c0eca8203 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:23.509662 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e96e0b-62f7-4893-867c-e41c0eca8203 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:23.511088 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e96e0b-62f7-4893-867c-e41c0eca8203 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:23.525608 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 333/682] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:23.527335 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb7a0c-f1a0-40a7-8fba-31b939d0bd27 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:23.527614 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb7a0c-f1a0-40a7-8fba-31b939d0bd27 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:23.527879 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb7a0c-f1a0-40a7-8fba-31b939d0bd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:23.539850 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d8ba3-9f02-42a7-bead-43ae0df2ce41 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:23.547324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-ccbb7a0c-f1a0-40a7-8fba-31b939d0bd27 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:23.548537 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-ccbb7a0c-f1a0-40a7-8fba-31b939d0bd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:23.576349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172c67a-5833-4ed1-a2d6-801e9ac55219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:23.576963 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172c67a-5833-4ed1-a2d6-801e9ac55219 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:23.577245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172c67a-5833-4ed1-a2d6-801e9ac55219 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:23.577514 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172c67a-5833-4ed1-a2d6-801e9ac55219 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:23.617727 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:23.636544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 334/683] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:54:23.637203 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb7a0c-f1a0-40a7-8fba-31b939d0bd27 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:23.646367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:23.681826 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172c67a-5833-4ed1-a2d6-801e9ac55219 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:23.683412 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172c67a-5833-4ed1-a2d6-801e9ac55219 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:23.805703 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb7a0c-f1a0-40a7-8fba-31b939d0bd27 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:23.842302 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f05e3-514e-4600-930f-7cea6a8150a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:23.843339 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f05e3-514e-4600-930f-7cea6a8150a4 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:23.843799 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f05e3-514e-4600-930f-7cea6a8150a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:23.844326 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f05e3-514e-4600-930f-7cea6a8150a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:23.850143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 350/684] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:54:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 01:54:23.852032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f05e3-514e-4600-930f-7cea6a8150a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:23.853697 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f05e3-514e-4600-930f-7cea6a8150a4 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:23.858212 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172c67a-5833-4ed1-a2d6-801e9ac55219 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:23.866878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-a32f05e3-514e-4600-930f-7cea6a8150a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:23.866878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:23.866878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:23.866878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:23.866878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:23.866878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:23.866878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:23.866878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:23.866878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:23.866878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:23.866878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:23.866878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:23.866878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:23.866878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:23.866878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:23.867822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:23.867822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:23.867822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:23.867822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:23.867822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:23.867822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:23.867822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:23.867822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:23.867822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:23.867822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:23.867822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:23.867822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:23.867822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:23.867822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:23.867822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:23.867822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:23.867822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:23.868755 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:23.868755 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:23.868755 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:23.868755 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:23.868755 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:23.868755 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:23.868755 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:23.868755 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:23.871255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 335/685] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:23.877033 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f05e3-514e-4600-930f-7cea6a8150a4 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:23.930692 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:23.965321 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:23.990924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172c67a-5833-4ed1-a2d6-801e9ac55219 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:24.005271 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f05e3-514e-4600-930f-7cea6a8150a4 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:24.030464 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044f510-51dd-458a-ab6c-b90f4909af41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:24.031099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044f510-51dd-458a-ab6c-b90f4909af41 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:24.031377 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044f510-51dd-458a-ab6c-b90f4909af41 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:24.031649 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044f510-51dd-458a-ab6c-b90f4909af41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:24.045097 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9044f510-51dd-458a-ab6c-b90f4909af41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:24.046642 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9044f510-51dd-458a-ab6c-b90f4909af41 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:24.052069 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4264835-51a0-45b4-aedc-b03c4f94f207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:24.052920 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4264835-51a0-45b4-aedc-b03c4f94f207 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:24.053388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4264835-51a0-45b4-aedc-b03c4f94f207 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:24.053847 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4264835-51a0-45b4-aedc-b03c4f94f207 None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:24.054789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4264835-51a0-45b4-aedc-b03c4f94f207 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:24.055269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4264835-51a0-45b4-aedc-b03c4f94f207 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:24.057034 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4264835-51a0-45b4-aedc-b03c4f94f207 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:24.062221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-9044f510-51dd-458a-ab6c-b90f4909af41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:24.062221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:24.062221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:24.062221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:24.062221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:24.062221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:24.062221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:24.062221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:24.062221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:24.062221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:24.062221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:24.062221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:24.062221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:24.062221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:24.062221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:24.062221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:24.063195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:24.063195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:24.063195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:24.063195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:24.063195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:24.063195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:24.063195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:24.063195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:24.063195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:24.063195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:24.063195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:24.063195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:24.063195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:24.063195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:24.063195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:24.063195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:24.063195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:24.064125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:24.064125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:24.064125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:24.068000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 351/686] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:24.068872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044f510-51dd-458a-ab6c-b90f4909af41 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:24.073706 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 336/687] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:54:24.079361 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4264835-51a0-45b4-aedc-b03c4f94f207 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:24.177722 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044f510-51dd-458a-ab6c-b90f4909af41 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:24.186575 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4264835-51a0-45b4-aedc-b03c4f94f207 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:24.200133 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:24.219370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:24.227292 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8cade-579f-479e-bfe9-d77623a78a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:24.227932 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8cade-579f-479e-bfe9-d77623a78a32 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:24.228214 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8cade-579f-479e-bfe9-d77623a78a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:24.228487 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8cade-579f-479e-bfe9-d77623a78a32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:24.230559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5f4aa-f3df-4cc0-b8b6-0219681ae3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:24.231425 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5f4aa-f3df-4cc0-b8b6-0219681ae3a0 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:24.231918 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5f4aa-f3df-4cc0-b8b6-0219681ae3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:24.232404 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5f4aa-f3df-4cc0-b8b6-0219681ae3a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:24.233257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5f4aa-f3df-4cc0-b8b6-0219681ae3a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:24.233770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5f4aa-f3df-4cc0-b8b6-0219681ae3a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:24.235527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5f4aa-f3df-4cc0-b8b6-0219681ae3a0 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:24.240614 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8cade-579f-479e-bfe9-d77623a78a32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:24.246983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-52f8cade-579f-479e-bfe9-d77623a78a32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:54:24.246983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:24.246983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:24.246983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:24.246983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:24.246983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:24.246983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:24.246983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:24.246983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:24.246983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:24.246983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:24.246983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:24.246983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:24.246983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:24.246983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:24.246983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:24.247940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:24.247940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:24.247940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:24.247940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:24.247940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:24.247940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:24.247940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:24.247940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:24.247940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:24.247940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:24.247940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:24.247940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:24.247940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:24.247940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:24.247940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:24.247940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:24.248808 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:24.248808 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:24.248808 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:24.248808 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:24.248808 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:24.248808 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:24.248808 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:24.248808 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:24.248808 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:24.248808 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:24.248808 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:54:24.248808 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:24.248808 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 352/688] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:54:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:24.249604 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8cade-579f-479e-bfe9-d77623a78a32 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:24.251815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 337/689] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:24.253634 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5f4aa-f3df-4cc0-b8b6-0219681ae3a0 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:24.355403 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5f4aa-f3df-4cc0-b8b6-0219681ae3a0 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:24.363873 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8cade-579f-479e-bfe9-d77623a78a32 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:24.392633 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf206b18-9170-435a-90e8-dab516be8b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:24.393625 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf206b18-9170-435a-90e8-dab516be8b13 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:24.394179 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf206b18-9170-435a-90e8-dab516be8b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:24.394682 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf206b18-9170-435a-90e8-dab516be8b13 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:24.395575 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf206b18-9170-435a-90e8-dab516be8b13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:24.396093 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf206b18-9170-435a-90e8-dab516be8b13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:24.397725 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf206b18-9170-435a-90e8-dab516be8b13 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:24.403176 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16389b-9185-4aa1-b547-78c49c7ed6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:24.403803 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16389b-9185-4aa1-b547-78c49c7ed6cb None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:24.404082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16389b-9185-4aa1-b547-78c49c7ed6cb None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:24.404351 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16389b-9185-4aa1-b547-78c49c7ed6cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:24.405078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16389b-9185-4aa1-b547-78c49c7ed6cb None admin] RBAC: Adding query filter params (role.id=e962a2a280224f9fbcd0fdc02262993d, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=bf86028f85af444380293d2387928eda) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:24.405382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16389b-9185-4aa1-b547-78c49c7ed6cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:24.407008 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16389b-9185-4aa1-b547-78c49c7ed6cb None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:24.410581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 338/690] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:54:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:54:24.411736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf206b18-9170-435a-90e8-dab516be8b13 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:24.414194 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 353/691] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:54:24 2022] GET /identity/v3/role_assignments?user.id=bf86028f85af444380293d2387928eda&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=e962a2a280224f9fbcd0fdc02262993d => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:24.416415 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16389b-9185-4aa1-b547-78c49c7ed6cb None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:24.445113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:24.483504 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:24.521055 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf206b18-9170-435a-90e8-dab516be8b13 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:24.545519 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16389b-9185-4aa1-b547-78c49c7ed6cb None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:24.558671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ba1f9-f9bd-4c51-906a-6bb9de326b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:24.559580 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ba1f9-f9bd-4c51-906a-6bb9de326b9c None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:24.560040 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ba1f9-f9bd-4c51-906a-6bb9de326b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:24.560459 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ba1f9-f9bd-4c51-906a-6bb9de326b9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:24.568475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ba1f9-f9bd-4c51-906a-6bb9de326b9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:24.570148 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ba1f9-f9bd-4c51-906a-6bb9de326b9c None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:24.583561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-a57ba1f9-f9bd-4c51-906a-6bb9de326b9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:24.583561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:24.583561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:24.583561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:24.583561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:24.583561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:24.583561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:24.583561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:24.583561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:24.583561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:24.583561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:24.583561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:24.583561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:24.583561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:24.583561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:24.584808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:24.584808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:24.584808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:24.584808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:24.584808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:24.584808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:24.584808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:24.584808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:24.584808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:24.584808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:24.584808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:24.584808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:24.584808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:24.584808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:24.584808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:24.584808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:24.584808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:24.585926 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:24.585926 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:24.585926 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:24.585926 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:24.585926 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:24.585926 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:24.585926 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:24.585926 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:24.588301 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 339/692] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:24.590384 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8702e-784d-4118-ad2e-0359de3696ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:24.591183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8702e-784d-4118-ad2e-0359de3696ca None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:24.591589 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8702e-784d-4118-ad2e-0359de3696ca None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:24.591986 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8702e-784d-4118-ad2e-0359de3696ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:24.594899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ba1f9-f9bd-4c51-906a-6bb9de326b9c None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:24.607714 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8702e-784d-4118-ad2e-0359de3696ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:24.609630 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8702e-784d-4118-ad2e-0359de3696ca None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:24.627647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-b3e8702e-784d-4118-ad2e-0359de3696ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:24.627647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:24.627647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:24.627647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:24.627647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:24.627647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:24.627647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:24.627647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:24.627647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:24.627647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:24.627647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:24.627647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:24.627647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:24.627647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:24.627647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:24.627647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:24.628739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:24.628739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:24.628739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:24.628739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:24.628739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:24.628739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:24.628739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:24.628739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:24.628739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:24.628739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:24.628739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:24.628739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:24.628739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:24.628739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:24.628739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:24.628739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:24.628739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:24.629669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:24.629669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:24.629669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:24.632354 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 354/693] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:24.638481 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8702e-784d-4118-ad2e-0359de3696ca None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:24.694737 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:24.703925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ba1f9-f9bd-4c51-906a-6bb9de326b9c None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:24.743938 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381df73-1c0b-4000-98fc-6ba9b08d301b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:24.744549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381df73-1c0b-4000-98fc-6ba9b08d301b None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:24.744828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381df73-1c0b-4000-98fc-6ba9b08d301b None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:24.745098 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381df73-1c0b-4000-98fc-6ba9b08d301b None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:24.745789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381df73-1c0b-4000-98fc-6ba9b08d301b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:24.746126 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381df73-1c0b-4000-98fc-6ba9b08d301b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:24.747565 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381df73-1c0b-4000-98fc-6ba9b08d301b None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:24.766716 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 340/694] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:24.770678 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381df73-1c0b-4000-98fc-6ba9b08d301b None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:24.811166 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8702e-784d-4118-ad2e-0359de3696ca None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:24.822455 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:24.854761 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be60da4-d086-4970-b461-eb6723e3263f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:24.855692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be60da4-d086-4970-b461-eb6723e3263f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:24.855979 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be60da4-d086-4970-b461-eb6723e3263f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:24.856561 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be60da4-d086-4970-b461-eb6723e3263f None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:24.857218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be60da4-d086-4970-b461-eb6723e3263f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:24.857521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be60da4-d086-4970-b461-eb6723e3263f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:24.860475 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be60da4-d086-4970-b461-eb6723e3263f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:24.878009 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 355/695] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:24.883685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be60da4-d086-4970-b461-eb6723e3263f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:24.884938 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381df73-1c0b-4000-98fc-6ba9b08d301b None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:24.929464 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95d083-5b6f-46b6-8734-f72d81594414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:24.930121 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95d083-5b6f-46b6-8734-f72d81594414 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:24.930401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95d083-5b6f-46b6-8734-f72d81594414 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:24.930674 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95d083-5b6f-46b6-8734-f72d81594414 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:24.931396 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95d083-5b6f-46b6-8734-f72d81594414 None admin] RBAC: Adding query filter params (role.id=e962a2a280224f9fbcd0fdc02262993d, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=21f481c1f60043e9b0e25ba49b30715f) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:24.931701 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95d083-5b6f-46b6-8734-f72d81594414 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:24.933129 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95d083-5b6f-46b6-8734-f72d81594414 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:24.939949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 341/696] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:54:24 2022] GET /identity/v3/role_assignments?user.id=21f481c1f60043e9b0e25ba49b30715f&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=e962a2a280224f9fbcd0fdc02262993d => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:25.014177 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be60da4-d086-4970-b461-eb6723e3263f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:25.068862 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df7bb6-23e4-428b-bba1-c7ca9c159419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:25.069521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df7bb6-23e4-428b-bba1-c7ca9c159419 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:25.069803 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df7bb6-23e4-428b-bba1-c7ca9c159419 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:25.070100 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df7bb6-23e4-428b-bba1-c7ca9c159419 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:25.078484 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df7bb6-23e4-428b-bba1-c7ca9c159419 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:25.079997 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df7bb6-23e4-428b-bba1-c7ca9c159419 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:25.099591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-c2df7bb6-23e4-428b-bba1-c7ca9c159419 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:54:25.099591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:25.099591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:25.099591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:25.099591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:25.099591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:25.099591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:25.099591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:25.099591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:25.099591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:25.099591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:25.099591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:25.099591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:25.099591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:25.099591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:25.099591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:25.100618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:25.100618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:25.100618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:25.100618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:25.100618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:25.100618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:25.100618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:25.100618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:25.100618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:25.100618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:25.100618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:25.100618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:25.100618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:25.100618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:25.100618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:25.100618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:25.100618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:25.100618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:25.102348 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:25.102348 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:25.102348 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:25.102348 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:25.102348 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:25.102348 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:25.102348 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:25.102348 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:25.102348 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:54:25.102348 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:25.102348 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 356/697] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Sep 14 01:54:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:25.106721 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95d083-5b6f-46b6-8734-f72d81594414 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:25.135249 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:25.153300 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3aab6-c74e-492e-88e3-407f84314cca None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:25.153632 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3aab6-c74e-492e-88e3-407f84314cca None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:25.153845 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e3aab6-c74e-492e-88e3-407f84314cca None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:25.155204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 357/698] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:54:25.160888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11554db-5322-4317-a47a-d4afbacc47af None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:25.161170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11554db-5322-4317-a47a-d4afbacc47af None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:25.161444 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11554db-5322-4317-a47a-d4afbacc47af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:25.194031 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-d11554db-5322-4317-a47a-d4afbacc47af None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:25.195212 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-d11554db-5322-4317-a47a-d4afbacc47af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:25.218349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:25.245657 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95d083-5b6f-46b6-8734-f72d81594414 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:25.283126 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 358/699] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:54:25.290082 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77dca99-d81c-4200-a4c0-9b667622875c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:25.290698 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dca99-d81c-4200-a4c0-9b667622875c None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:25.291024 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dca99-d81c-4200-a4c0-9b667622875c None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:25.291303 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dca99-d81c-4200-a4c0-9b667622875c None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:25.292035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77dca99-d81c-4200-a4c0-9b667622875c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:25.292335 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77dca99-d81c-4200-a4c0-9b667622875c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:25.293763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77dca99-d81c-4200-a4c0-9b667622875c None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:25.314785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 342/700] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Sep 14 01:54:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:25.321247 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11554db-5322-4317-a47a-d4afbacc47af None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:25.442993 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11554db-5322-4317-a47a-d4afbacc47af None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:25.492140 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b60cd8-31d0-49f1-9086-96f018699e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:25.492749 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b60cd8-31d0-49f1-9086-96f018699e11 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:25.493024 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b60cd8-31d0-49f1-9086-96f018699e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:25.493293 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b60cd8-31d0-49f1-9086-96f018699e11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:25.529568 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:25.572028 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77dca99-d81c-4200-a4c0-9b667622875c None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:25.595562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b60cd8-31d0-49f1-9086-96f018699e11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:25.598158 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b60cd8-31d0-49f1-9086-96f018699e11 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:25.693973 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:25.753496 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77dca99-d81c-4200-a4c0-9b667622875c None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:25.775891 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 359/701] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:54:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:54:25.781070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b60cd8-31d0-49f1-9086-96f018699e11 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:25.817604 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42c83e-4b1f-4437-834e-b47acebf87a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:25.818637 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42c83e-4b1f-4437-834e-b47acebf87a1 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:25.819138 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42c83e-4b1f-4437-834e-b47acebf87a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:25.819613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42c83e-4b1f-4437-834e-b47acebf87a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:25.835932 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42c83e-4b1f-4437-834e-b47acebf87a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:25.838548 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42c83e-4b1f-4437-834e-b47acebf87a1 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:25.871277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-bb42c83e-4b1f-4437-834e-b47acebf87a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:25.871277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:25.871277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:25.871277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:25.871277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:25.871277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:25.871277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:25.871277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:25.871277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:25.871277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:25.871277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:25.871277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:25.871277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:25.871277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:25.871277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:25.871277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:25.872341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:25.872341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:25.872341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:25.872341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:25.872341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:25.872341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:25.872341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:25.872341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:25.872341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:25.872341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:25.872341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:25.872341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:25.872341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:25.872341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:25.872341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:25.872341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:25.872341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:25.873472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:25.873472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:25.873472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:25.878831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 343/702] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:25.886797 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42c83e-4b1f-4437-834e-b47acebf87a1 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:25.898777 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:25.958643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b60cd8-31d0-49f1-9086-96f018699e11 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:26.008051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97e952-230e-414b-af4e-0b19e0febf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:26.009619 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97e952-230e-414b-af4e-0b19e0febf4f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:26.010249 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97e952-230e-414b-af4e-0b19e0febf4f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:26.010697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97e952-230e-414b-af4e-0b19e0febf4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:26.024739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97e952-230e-414b-af4e-0b19e0febf4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:26.027241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97e952-230e-414b-af4e-0b19e0febf4f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:26.030028 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42c83e-4b1f-4437-834e-b47acebf87a1 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:26.048332 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-dd97e952-230e-414b-af4e-0b19e0febf4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:26.048332 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:26.048332 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:26.048332 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:26.048332 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:26.048332 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:26.048332 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:26.048332 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:26.048332 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:26.048332 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:26.048332 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:26.048332 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:26.048332 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:26.048332 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:26.048332 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:26.048332 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:26.049460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:26.049460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:26.049460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:26.049460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:26.049460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:26.049460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:26.049460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:26.049460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:26.049460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:26.049460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:26.049460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:26.049460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:26.049460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:26.049460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:26.049460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:26.049460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:26.049460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:26.050420 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:26.050420 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:26.050420 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:26.054309 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 360/703] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:26.055838 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44affa88-a9b1-4bbc-9591-8bd0526bd6ea None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:26.056318 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44affa88-a9b1-4bbc-9591-8bd0526bd6ea None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:26.057052 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44affa88-a9b1-4bbc-9591-8bd0526bd6ea None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:26.059360 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 361/704] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:26.060263 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44affa88-a9b1-4bbc-9591-8bd0526bd6ea None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:26.072409 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:26.090694 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877698cb-6ec8-4298-8b35-446bc75fe828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:26.091311 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877698cb-6ec8-4298-8b35-446bc75fe828 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:26.091591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877698cb-6ec8-4298-8b35-446bc75fe828 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:26.092275 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877698cb-6ec8-4298-8b35-446bc75fe828 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:26.092506 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877698cb-6ec8-4298-8b35-446bc75fe828 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:26.092809 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877698cb-6ec8-4298-8b35-446bc75fe828 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:26.094207 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877698cb-6ec8-4298-8b35-446bc75fe828 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:26.110021 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 344/705] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:26.110021 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6e070-f201-4247-be50-a53ebc683040 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:26.110021 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6e070-f201-4247-be50-a53ebc683040 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:26.110021 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6e070-f201-4247-be50-a53ebc683040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:26.127513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-8ae6e070-f201-4247-be50-a53ebc683040 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:26.128631 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-8ae6e070-f201-4247-be50-a53ebc683040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:26.198611 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 345/706] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:54:26.201376 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44affa88-a9b1-4bbc-9591-8bd0526bd6ea None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:26.206807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6e070-f201-4247-be50-a53ebc683040 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:26.213231 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:26.263696 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329eb611-8c66-46b1-9f9b-bf69ab55b8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:26.264314 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329eb611-8c66-46b1-9f9b-bf69ab55b8dc None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:26.264594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329eb611-8c66-46b1-9f9b-bf69ab55b8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:26.264863 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329eb611-8c66-46b1-9f9b-bf69ab55b8dc None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:26.265501 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329eb611-8c66-46b1-9f9b-bf69ab55b8dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:26.265804 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329eb611-8c66-46b1-9f9b-bf69ab55b8dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:26.267497 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329eb611-8c66-46b1-9f9b-bf69ab55b8dc None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:26.298485 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 362/707] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:26.299946 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329eb611-8c66-46b1-9f9b-bf69ab55b8dc None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:26.427208 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329eb611-8c66-46b1-9f9b-bf69ab55b8dc None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:26.456176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6e070-f201-4247-be50-a53ebc683040 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:26.472033 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caef273-41f3-4acc-96ab-6dd6898492bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:26.473257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caef273-41f3-4acc-96ab-6dd6898492bf None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:26.473733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caef273-41f3-4acc-96ab-6dd6898492bf None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:26.476240 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caef273-41f3-4acc-96ab-6dd6898492bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:26.487917 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:26.491290 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caef273-41f3-4acc-96ab-6dd6898492bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:26.494506 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caef273-41f3-4acc-96ab-6dd6898492bf None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:26.516098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-7caef273-41f3-4acc-96ab-6dd6898492bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:26.516098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:26.516098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:26.516098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:26.516098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:26.516098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:26.516098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:26.516098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:26.516098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:26.516098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:26.516098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:26.516098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:26.516098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:26.516098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:26.516098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:26.517070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:26.517070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:26.517070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:26.517070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:26.517070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:26.517070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:26.517070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:26.517070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:26.517070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:26.517070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:26.517070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:26.517070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:26.517070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:26.517070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:26.517070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:26.517070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:26.517070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:26.518123 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:26.518123 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:26.518123 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:26.518123 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:26.518123 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:26.518123 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:26.518123 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:26.518123 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:26.523215 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 363/708] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:26.523215 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caef273-41f3-4acc-96ab-6dd6898492bf None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:26.530645 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839942d1-8ef0-4691-a3a7-9af06ef2d322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:26.531495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839942d1-8ef0-4691-a3a7-9af06ef2d322 None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:26.532753 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839942d1-8ef0-4691-a3a7-9af06ef2d322 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:26.533747 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839942d1-8ef0-4691-a3a7-9af06ef2d322 None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:26.536019 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839942d1-8ef0-4691-a3a7-9af06ef2d322 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:26.537602 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839942d1-8ef0-4691-a3a7-9af06ef2d322 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:26.572289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:26.581145 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 346/709] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 14 01:54:26 2022] POST /identity/v3/users => generated 254 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:54:26.583542 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839942d1-8ef0-4691-a3a7-9af06ef2d322 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:26.724312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caef273-41f3-4acc-96ab-6dd6898492bf None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:26.769132 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c2d55-56e5-40cf-9a6d-846216b7d501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:26.770220 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c2d55-56e5-40cf-9a6d-846216b7d501 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:26.770701 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c2d55-56e5-40cf-9a6d-846216b7d501 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:26.771146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c2d55-56e5-40cf-9a6d-846216b7d501 None admin] PATH_INFO: `/v3/projects` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:26.772027 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c2d55-56e5-40cf-9a6d-846216b7d501 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:26.772562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c2d55-56e5-40cf-9a6d-846216b7d501 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:26.774328 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c2d55-56e5-40cf-9a6d-846216b7d501 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:26.795104 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 364/710] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:26.800699 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839942d1-8ef0-4691-a3a7-9af06ef2d322 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:26.801879 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c2d55-56e5-40cf-9a6d-846216b7d501 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:26.836098 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b88a7-dee7-42e8-b5a8-5d01c00d10de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:26.836984 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b88a7-dee7-42e8-b5a8-5d01c00d10de None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:26.837343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b88a7-dee7-42e8-b5a8-5d01c00d10de None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:26.837720 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b88a7-dee7-42e8-b5a8-5d01c00d10de None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:26.838440 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b88a7-dee7-42e8-b5a8-5d01c00d10de None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:26.839899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b88a7-dee7-42e8-b5a8-5d01c00d10de None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:26.852199 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-1a6b88a7-dee7-42e8-b5a8-5d01c00d10de None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 01:54:26.852199 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:26.852199 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:54:26.852199 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:54:26.852199 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:54:26.852199 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:54:26.852199 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:54:26.852199 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:26.852199 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 01:54:26.852199 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:26.852199 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:54:26.852199 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:26.852199 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:26.852199 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:26.852199 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:26.852199 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:26.852199 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:26.853486 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:26.853486 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:26.853486 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:26.853486 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:26.853486 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:26.853486 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:26.853486 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:54:26.853486 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:54:26.853486 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:54:26.853486 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:54:26.853486 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:26.853486 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:26.853486 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:26.853486 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:26.853486 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:26.853486 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:26.854444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:26.854444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:26.854444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:26.854444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:26.854444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:26.854444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:26.854444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:26.854444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:26.854444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:26.854444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:26.854444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:54:26.854444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:26.854444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 01:54:26.854444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:26.855205 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 347/711] 10.222.0.15 () {56 vars in 1129 bytes} [Wed Sep 14 01:54:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:26.860951 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b88a7-dee7-42e8-b5a8-5d01c00d10de None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:26.902255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:26.907680 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:26.925620 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c2d55-56e5-40cf-9a6d-846216b7d501 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:26.968670 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b82b3-7ba8-4fa4-ac3a-cdc1524f715e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:26.969307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b82b3-7ba8-4fa4-ac3a-cdc1524f715e None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:26.969751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b82b3-7ba8-4fa4-ac3a-cdc1524f715e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:26.970240 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b82b3-7ba8-4fa4-ac3a-cdc1524f715e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:26.971528 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b82b3-7ba8-4fa4-ac3a-cdc1524f715e None admin] RBAC: Adding query filter params (role.id=aa420c2fb1ab4bb3b05b004a4d89779a, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=f721a3a4eca941989685448a87bb9a19) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:26.972052 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b82b3-7ba8-4fa4-ac3a-cdc1524f715e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:26.974177 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b82b3-7ba8-4fa4-ac3a-cdc1524f715e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:26.981908 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 365/712] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:54:26 2022] GET /identity/v3/role_assignments?user.id=f721a3a4eca941989685448a87bb9a19&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=aa420c2fb1ab4bb3b05b004a4d89779a => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:26.992076 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b88a7-dee7-42e8-b5a8-5d01c00d10de None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:27.039801 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d41a0-1e08-40d4-bcda-b9e068bb2fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:27.040439 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d41a0-1e08-40d4-bcda-b9e068bb2fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:27.040716 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d41a0-1e08-40d4-bcda-b9e068bb2fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:27.040985 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d41a0-1e08-40d4-bcda-b9e068bb2fb7 None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:27.041638 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d41a0-1e08-40d4-bcda-b9e068bb2fb7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:27.041936 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d41a0-1e08-40d4-bcda-b9e068bb2fb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:27.043400 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d41a0-1e08-40d4-bcda-b9e068bb2fb7 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:27.049714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 348/713] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Sep 14 01:54:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:27.055182 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b82b3-7ba8-4fa4-ac3a-cdc1524f715e None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:27.176205 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b82b3-7ba8-4fa4-ac3a-cdc1524f715e None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:27.212668 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a1eca-53f2-474a-bfe3-aac00f0df60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:27.213275 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a1eca-53f2-474a-bfe3-aac00f0df60b None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:27.213548 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a1eca-53f2-474a-bfe3-aac00f0df60b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:27.213815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a1eca-53f2-474a-bfe3-aac00f0df60b None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:27.214465 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a1eca-53f2-474a-bfe3-aac00f0df60b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:27.214761 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a1eca-53f2-474a-bfe3-aac00f0df60b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:27.216153 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a1eca-53f2-474a-bfe3-aac00f0df60b None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:27.223118 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 366/714] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Sep 14 01:54:27 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:27.441572 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4356b06-7bab-4304-9895-42cd49b553c4 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:27.441903 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4356b06-7bab-4304-9895-42cd49b553c4 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:27.442303 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4356b06-7bab-4304-9895-42cd49b553c4 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:27.444885 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 349/715] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:27.452106 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae49921-b965-4606-bfc4-5247e6cc4cac None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:27.452435 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae49921-b965-4606-bfc4-5247e6cc4cac None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:27.452663 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae49921-b965-4606-bfc4-5247e6cc4cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:27.477819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-1ae49921-b965-4606-bfc4-5247e6cc4cac None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:27.479076 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-1ae49921-b965-4606-bfc4-5247e6cc4cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:27.494472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:27.579364 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e8ce0-a3c3-49a4-8b9f-dd24ce0e4293 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:27.579675 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e8ce0-a3c3-49a4-8b9f-dd24ce0e4293 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:27.580117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e8ce0-a3c3-49a4-8b9f-dd24ce0e4293 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:27.582412 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 367/717] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:54:27.582539 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 350/717] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:27.587337 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e343a-d7f5-4392-8ae0-4bcd4cfeebd1 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:27.587614 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e343a-d7f5-4392-8ae0-4bcd4cfeebd1 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:27.587931 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e343a-d7f5-4392-8ae0-4bcd4cfeebd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:27.605648 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-9d7e343a-d7f5-4392-8ae0-4bcd4cfeebd1 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:27.606911 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-9d7e343a-d7f5-4392-8ae0-4bcd4cfeebd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:27.708648 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 368/718] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:54:27.959468 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e8ce0-a3c3-49a4-8b9f-dd24ce0e4293 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:28.089206 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:28.096096 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e8ce0-a3c3-49a4-8b9f-dd24ce0e4293 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:28.134715 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bdb2a-7d7a-4129-8f09-ad17ec5ddb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:28.135767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bdb2a-7d7a-4129-8f09-ad17ec5ddb21 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:28.136241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bdb2a-7d7a-4129-8f09-ad17ec5ddb21 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:28.136701 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bdb2a-7d7a-4129-8f09-ad17ec5ddb21 None admin] PATH_INFO: `/v3/services/network` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:28.136845 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e343a-d7f5-4392-8ae0-4bcd4cfeebd1 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:28.137614 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04bdb2a-7d7a-4129-8f09-ad17ec5ddb21 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:28.139256 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04bdb2a-7d7a-4129-8f09-ad17ec5ddb21 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:28.152071 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-d04bdb2a-7d7a-4129-8f09-ad17ec5ddb21 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 01:54:28.152071 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:28.152071 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:54:28.152071 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:54:28.152071 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:54:28.152071 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:54:28.152071 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:54:28.152071 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:28.152071 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 01:54:28.152071 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:28.152071 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:54:28.152071 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:28.152071 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:28.152071 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:28.152071 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:28.152071 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:28.152071 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:28.153099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:28.153099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:28.153099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:28.153099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:28.153099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:28.153099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:28.153099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:54:28.153099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:54:28.153099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:54:28.153099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:54:28.153099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:28.153099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:28.153099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:28.153099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:28.153099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:28.153099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:28.153952 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:28.153952 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:28.153952 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:28.153952 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:28.153952 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:28.153952 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:28.153952 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:28.153952 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:28.153952 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:28.153952 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:28.153952 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:54:28.153952 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:28.153952 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 01:54:28.153952 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:28.156170 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 351/719] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:27 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:28.161839 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bdb2a-7d7a-4129-8f09-ad17ec5ddb21 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:28.282033 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:28.323901 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e343a-d7f5-4392-8ae0-4bcd4cfeebd1 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:28.328205 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bdb2a-7d7a-4129-8f09-ad17ec5ddb21 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:28.375526 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf105a-f056-41be-acb9-0b9aa05c7baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:28.376933 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf105a-f056-41be-acb9-0b9aa05c7baa None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:28.377532 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf105a-f056-41be-acb9-0b9aa05c7baa None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:28.378075 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf105a-f056-41be-acb9-0b9aa05c7baa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:28.404023 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf105a-f056-41be-acb9-0b9aa05c7baa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:28.409729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-29bf105a-f056-41be-acb9-0b9aa05c7baa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:54:28.409729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:28.409729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:28.409729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:28.409729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:28.409729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:28.409729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:28.409729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:28.409729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:28.409729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:28.409729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:28.409729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:28.409729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:28.409729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:28.409729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:28.409729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:28.410719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:28.410719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:28.410719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:28.410719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:28.410719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:28.410719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:28.410719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:28.410719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:28.410719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:28.410719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:28.410719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:28.410719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:28.410719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:28.410719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:28.410719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:28.410719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:28.411571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:28.411571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:28.411571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:28.411571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:28.411571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:28.411571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:28.411571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:28.411571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:28.411571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:28.411571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:28.411571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:54:28.411571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:28.414305 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c8d88-2b01-46f9-916b-c67793b2ae44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:28.415149 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c8d88-2b01-46f9-916b-c67793b2ae44 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:28.415625 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c8d88-2b01-46f9-916b-c67793b2ae44 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:28.416092 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c8d88-2b01-46f9-916b-c67793b2ae44 None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:28.416937 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c8d88-2b01-46f9-916b-c67793b2ae44 None admin] RBAC: Adding query filter params (name=network) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:28.417489 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c8d88-2b01-46f9-916b-c67793b2ae44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:28.419091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c8d88-2b01-46f9-916b-c67793b2ae44 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:28.421918 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 369/720] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:54:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:28.432316 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf105a-f056-41be-acb9-0b9aa05c7baa None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:28.433404 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 352/721] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 14 01:54:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:54:28.436878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c8d88-2b01-46f9-916b-c67793b2ae44 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:28.503431 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:28.555354 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf105a-f056-41be-acb9-0b9aa05c7baa None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:28.631794 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d507f4-a6e5-4778-83ec-e546c4e8304f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:28.632644 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d507f4-a6e5-4778-83ec-e546c4e8304f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:28.633094 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d507f4-a6e5-4778-83ec-e546c4e8304f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:28.633505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d507f4-a6e5-4778-83ec-e546c4e8304f None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:28.634294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c8d88-2b01-46f9-916b-c67793b2ae44 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:28.634753 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d507f4-a6e5-4778-83ec-e546c4e8304f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:28.634753 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d507f4-a6e5-4778-83ec-e546c4e8304f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:28.636080 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d507f4-a6e5-4778-83ec-e546c4e8304f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:28.642283 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:28.652596 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 370/722] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:54:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:28.668461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d507f4-a6e5-4778-83ec-e546c4e8304f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:28.674936 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642acfce-019d-416b-91ef-203e5b8bfdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:28.675541 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642acfce-019d-416b-91ef-203e5b8bfdd8 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:28.675820 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642acfce-019d-416b-91ef-203e5b8bfdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:28.676091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642acfce-019d-416b-91ef-203e5b8bfdd8 None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:28.676714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642acfce-019d-416b-91ef-203e5b8bfdd8 None admin] RBAC: Adding query filter params (type=network) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:28.677016 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642acfce-019d-416b-91ef-203e5b8bfdd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:28.678412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642acfce-019d-416b-91ef-203e5b8bfdd8 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:28.684206 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 353/723] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 14 01:54:28 2022] GET /identity/v3/services?type=network => generated 124 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:28.815276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d507f4-a6e5-4778-83ec-e546c4e8304f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:28.857508 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07189ebc-1850-4cf2-b82e-518d3087424d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:28.858456 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07189ebc-1850-4cf2-b82e-518d3087424d None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:28.858901 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07189ebc-1850-4cf2-b82e-518d3087424d None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:28.859338 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07189ebc-1850-4cf2-b82e-518d3087424d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:28.871171 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07189ebc-1850-4cf2-b82e-518d3087424d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:28.872899 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07189ebc-1850-4cf2-b82e-518d3087424d None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:28.888276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-07189ebc-1850-4cf2-b82e-518d3087424d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:28.888276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:28.888276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:28.888276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:28.888276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:28.888276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:28.888276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:28.888276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:28.888276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:28.888276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:28.888276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:28.888276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:28.888276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:28.888276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:28.888276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:28.888276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:28.889444 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:28.889444 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:28.889444 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:28.889444 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:28.889444 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:28.889444 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:28.889444 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:28.889444 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:28.889444 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:28.889444 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:28.889444 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:28.889444 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:28.889444 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:28.889444 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:28.889444 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:28.889444 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:28.889444 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:28.890668 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:28.890668 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:28.890668 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:28.892381 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 371/724] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:28.898312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642acfce-019d-416b-91ef-203e5b8bfdd8 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:29.001323 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:29.015073 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642acfce-019d-416b-91ef-203e5b8bfdd8 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:29.059581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f60bac-5b68-4656-9d47-0bd44d594ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:29.060555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f60bac-5b68-4656-9d47-0bd44d594ac9 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:29.061055 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f60bac-5b68-4656-9d47-0bd44d594ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:29.061533 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f60bac-5b68-4656-9d47-0bd44d594ac9 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:29.062434 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f60bac-5b68-4656-9d47-0bd44d594ac9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:29.062956 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f60bac-5b68-4656-9d47-0bd44d594ac9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:29.065067 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f60bac-5b68-4656-9d47-0bd44d594ac9 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:29.082970 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 354/725] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:29.091099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07189ebc-1850-4cf2-b82e-518d3087424d None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:29.171680 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:29.229404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07189ebc-1850-4cf2-b82e-518d3087424d None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:29.264424 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea9999-0049-48d6-8c3c-4e3ff6893814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:29.265060 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea9999-0049-48d6-8c3c-4e3ff6893814 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:29.265330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea9999-0049-48d6-8c3c-4e3ff6893814 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:29.265595 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea9999-0049-48d6-8c3c-4e3ff6893814 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:29.274399 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea9999-0049-48d6-8c3c-4e3ff6893814 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:29.275877 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea9999-0049-48d6-8c3c-4e3ff6893814 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:29.288394 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-98ea9999-0049-48d6-8c3c-4e3ff6893814 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:54:29.288394 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:29.288394 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:29.288394 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:29.288394 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:29.288394 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:29.288394 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:29.288394 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:29.288394 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:29.288394 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:29.288394 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:29.288394 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:29.288394 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:29.288394 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:29.288394 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:29.288394 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:29.289452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:29.289452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:29.289452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:29.289452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:29.289452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:29.289452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:29.289452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:29.289452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:29.289452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:29.289452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:29.289452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:29.289452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:29.289452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:29.289452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:29.289452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:29.289452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:29.289452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:29.289452 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:29.290457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:29.290457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:29.290457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:29.290457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:29.290457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:29.290457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:29.290457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:29.290457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:29.290457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:54:29.290457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:29.290457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 372/726] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:54:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:29.299705 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f60bac-5b68-4656-9d47-0bd44d594ac9 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:29.425588 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f60bac-5b68-4656-9d47-0bd44d594ac9 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:29.430087 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee82c7-f0b6-4320-ab07-8dbe3db39435 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:29.430643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee82c7-f0b6-4320-ab07-8dbe3db39435 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:29.431119 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee82c7-f0b6-4320-ab07-8dbe3db39435 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:29.436625 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 373/727] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:29.443315 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63438a28-d1f0-4991-bcbb-b44ddc7d6136 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:29.443888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63438a28-d1f0-4991-bcbb-b44ddc7d6136 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:29.444367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63438a28-d1f0-4991-bcbb-b44ddc7d6136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:29.460305 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e82245-a1c7-4bf5-8345-08fdcee97811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:29.461167 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e82245-a1c7-4bf5-8345-08fdcee97811 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:29.461592 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e82245-a1c7-4bf5-8345-08fdcee97811 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:29.462027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e82245-a1c7-4bf5-8345-08fdcee97811 None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:29.462896 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e82245-a1c7-4bf5-8345-08fdcee97811 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:29.463344 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e82245-a1c7-4bf5-8345-08fdcee97811 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:29.464976 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e82245-a1c7-4bf5-8345-08fdcee97811 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:29.490296 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-63438a28-d1f0-4991-bcbb-b44ddc7d6136 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:29.491891 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-63438a28-d1f0-4991-bcbb-b44ddc7d6136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:29.494086 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 355/728] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Sep 14 01:54:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:29.495666 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e82245-a1c7-4bf5-8345-08fdcee97811 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:29.507316 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:29.608170 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e82245-a1c7-4bf5-8345-08fdcee97811 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:29.636626 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 374/729] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:54:29.653626 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72350e12-3f3a-4d73-b64e-ae84a49f5232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:29.654790 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72350e12-3f3a-4d73-b64e-ae84a49f5232 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:29.655274 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72350e12-3f3a-4d73-b64e-ae84a49f5232 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:29.655690 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72350e12-3f3a-4d73-b64e-ae84a49f5232 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:29.701139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3fd95-01ba-4e0f-8207-08a89a3083b5 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:29.701429 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3fd95-01ba-4e0f-8207-08a89a3083b5 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:29.701708 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3fd95-01ba-4e0f-8207-08a89a3083b5 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:29.703311 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 375/730] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:29.721573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da1575-a707-41f4-9cfb-f62189de90cd None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:29.726199 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da1575-a707-41f4-9cfb-f62189de90cd None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:29.726445 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da1575-a707-41f4-9cfb-f62189de90cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:29.742361 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72350e12-3f3a-4d73-b64e-ae84a49f5232 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:29.746619 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72350e12-3f3a-4d73-b64e-ae84a49f5232 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:29.766333 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:29.769497 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-17da1575-a707-41f4-9cfb-f62189de90cd None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:29.771486 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-17da1575-a707-41f4-9cfb-f62189de90cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:29.842333 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:29.896405 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 376/731] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:54:29.925212 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 356/732] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:54:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 01:54:29.931080 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da1575-a707-41f4-9cfb-f62189de90cd None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:29.946527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72350e12-3f3a-4d73-b64e-ae84a49f5232 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:30.055542 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da1575-a707-41f4-9cfb-f62189de90cd None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:30.099191 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb211e3-f44e-4441-8f56-7dbe3a93f131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:30.100129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb211e3-f44e-4441-8f56-7dbe3a93f131 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:30.100577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72350e12-3f3a-4d73-b64e-ae84a49f5232 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:30.100718 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb211e3-f44e-4441-8f56-7dbe3a93f131 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:30.100855 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb211e3-f44e-4441-8f56-7dbe3a93f131 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:30.113613 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb211e3-f44e-4441-8f56-7dbe3a93f131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:30.115278 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb211e3-f44e-4441-8f56-7dbe3a93f131 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:30.134222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-9eb211e3-f44e-4441-8f56-7dbe3a93f131 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:30.134222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:30.134222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:30.134222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:30.134222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:30.134222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:30.134222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:30.134222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:30.134222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:30.134222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:30.134222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:30.134222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:30.134222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:30.134222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:30.134222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:30.134222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:30.135186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:30.135186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:30.135186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:30.135186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:30.135186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:30.135186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:30.135186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:30.135186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:30.135186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:30.135186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:30.135186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:30.135186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:30.135186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:30.135186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:30.135186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:30.135186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:30.135186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:30.136509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:30.136509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:30.136509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:30.136509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 377/733] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:30.136741 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8f81b-5ae7-420e-99a9-eb890fd30ee3 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:30.141275 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8f81b-5ae7-420e-99a9-eb890fd30ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:30.141405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d8df3-56fb-4094-b770-0f29aa59d591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:30.141405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d8df3-56fb-4094-b770-0f29aa59d591 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:30.141405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d8df3-56fb-4094-b770-0f29aa59d591 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:30.141405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d8df3-56fb-4094-b770-0f29aa59d591 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:30.142802 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8f81b-5ae7-420e-99a9-eb890fd30ee3 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:30.144643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 378/734] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:30.145641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8f81b-5ae7-420e-99a9-eb890fd30ee3 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:30.146419 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:30.154910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d8df3-56fb-4094-b770-0f29aa59d591 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:30.161147 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-296d8df3-56fb-4094-b770-0f29aa59d591 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:30.161147 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:30.161147 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:30.161147 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:30.161147 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:30.161147 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:30.161147 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:30.161147 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:30.161147 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:30.161147 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:30.161147 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:30.161147 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:30.161147 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:30.161147 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:30.161147 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:30.161147 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:30.162330 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:30.162330 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:30.162330 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:30.162330 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:30.162330 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:30.162330 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:30.162330 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:30.162330 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:30.162330 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:30.162330 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:30.162330 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:30.162330 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:30.162330 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:30.162330 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:30.162330 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:30.162330 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:30.163257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:30.163257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:30.163257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:30.163257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:30.163257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:30.163257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:30.163257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:30.163257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:30.163257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:30.163257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:30.163257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:30.163257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:30.168399 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 357/735] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:54:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:30.169888 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2333a76-f166-4261-a2f6-f9e7d16684d8 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:30.170267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2333a76-f166-4261-a2f6-f9e7d16684d8 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:30.170613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2333a76-f166-4261-a2f6-f9e7d16684d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:30.178420 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:30.190375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-c2333a76-f166-4261-a2f6-f9e7d16684d8 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:30.191858 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-c2333a76-f166-4261-a2f6-f9e7d16684d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:30.270828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 358/736] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:54:30.271029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8f81b-5ae7-420e-99a9-eb890fd30ee3 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:30.271228 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2333a76-f166-4261-a2f6-f9e7d16684d8 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:30.328577 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d590d-c4b8-4b1a-a4ea-c0ec0a06a92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:30.330025 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d590d-c4b8-4b1a-a4ea-c0ec0a06a92a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:30.330488 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d590d-c4b8-4b1a-a4ea-c0ec0a06a92a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:30.331382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d590d-c4b8-4b1a-a4ea-c0ec0a06a92a None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:30.332522 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d590d-c4b8-4b1a-a4ea-c0ec0a06a92a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:30.333006 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d590d-c4b8-4b1a-a4ea-c0ec0a06a92a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:30.335373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d590d-c4b8-4b1a-a4ea-c0ec0a06a92a None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:30.352361 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 379/737] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:30.353658 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d590d-c4b8-4b1a-a4ea-c0ec0a06a92a None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:30.391600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:30.396016 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2333a76-f166-4261-a2f6-f9e7d16684d8 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:30.432937 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a9e76-aeda-4ed5-8738-e22387eb911c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:30.433840 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a9e76-aeda-4ed5-8738-e22387eb911c None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:30.434327 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a9e76-aeda-4ed5-8738-e22387eb911c None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:30.434880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a9e76-aeda-4ed5-8738-e22387eb911c None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:30.435724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a9e76-aeda-4ed5-8738-e22387eb911c None admin] RBAC: Adding query filter params (name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:30.436207 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a9e76-aeda-4ed5-8738-e22387eb911c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:30.437745 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a9e76-aeda-4ed5-8738-e22387eb911c None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:30.447176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 359/738] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 14 01:54:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:30.448264 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a9e76-aeda-4ed5-8738-e22387eb911c None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:30.454972 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:30.475065 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d590d-c4b8-4b1a-a4ea-c0ec0a06a92a None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:30.510248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b164da7-fd6c-47e7-8be4-4b834299418b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:30.510953 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b164da7-fd6c-47e7-8be4-4b834299418b None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:30.511368 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b164da7-fd6c-47e7-8be4-4b834299418b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:30.511630 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b164da7-fd6c-47e7-8be4-4b834299418b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:30.522106 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b164da7-fd6c-47e7-8be4-4b834299418b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:30.527477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-3b164da7-fd6c-47e7-8be4-4b834299418b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:54:30.527477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:30.527477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:30.527477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:30.527477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:30.527477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:30.527477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:30.527477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:30.527477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:30.527477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:30.527477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:30.527477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:30.527477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:30.527477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:30.527477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:30.527477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:30.528457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:30.528457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:30.528457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:30.528457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:30.528457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:30.528457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:30.528457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:30.528457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:30.528457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:30.528457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:30.528457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:30.528457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:30.528457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:30.528457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:30.528457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:30.528457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:30.529345 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:30.529345 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:30.529345 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:30.529345 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:30.529345 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:30.529345 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:30.529345 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:30.529345 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:30.529345 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:30.529345 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:30.529345 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:54:30.529345 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:30.534261 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 380/739] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:54:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:30.535164 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b164da7-fd6c-47e7-8be4-4b834299418b None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:30.560805 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a9e76-aeda-4ed5-8738-e22387eb911c None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:30.595772 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb4dea-fba3-4ead-82c0-58c316906694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:30.596399 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb4dea-fba3-4ead-82c0-58c316906694 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:30.596675 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb4dea-fba3-4ead-82c0-58c316906694 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:30.596951 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb4dea-fba3-4ead-82c0-58c316906694 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:30.603481 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb4dea-fba3-4ead-82c0-58c316906694 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:30.604913 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb4dea-fba3-4ead-82c0-58c316906694 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:30.616519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-67cb4dea-fba3-4ead-82c0-58c316906694 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:30.616519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:30.616519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:30.616519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:30.616519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:30.616519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:30.616519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:30.616519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:30.616519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:30.616519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:30.616519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:30.616519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:30.616519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:30.616519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:30.616519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:30.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:30.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:30.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:30.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:30.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:30.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:30.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:30.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:30.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:30.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:30.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:30.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:30.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:30.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:30.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:30.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:30.617479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:30.623400 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:30.623400 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:30.623400 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:30.623400 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:30.623400 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:30.623400 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:30.623400 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:30.623400 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:30.623400 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 360/740] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:30.623400 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb4dea-fba3-4ead-82c0-58c316906694 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:30.645925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b164da7-fd6c-47e7-8be4-4b834299418b None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:30.649829 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:30.684881 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fde6e-65f2-4f38-a982-9b65717c4da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:30.685440 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fde6e-65f2-4f38-a982-9b65717c4da6 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:30.685720 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fde6e-65f2-4f38-a982-9b65717c4da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:30.686018 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fde6e-65f2-4f38-a982-9b65717c4da6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:30.696402 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fde6e-65f2-4f38-a982-9b65717c4da6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:30.697849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fde6e-65f2-4f38-a982-9b65717c4da6 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:30.699605 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:30.713090 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-a85fde6e-65f2-4f38-a982-9b65717c4da6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:30.713090 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:30.713090 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:30.713090 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:30.713090 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:30.713090 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:30.713090 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:30.713090 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:30.713090 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:30.713090 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:30.713090 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:30.713090 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:30.713090 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:30.713090 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:30.713090 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:30.713090 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:30.714143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:30.714143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:30.714143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:30.714143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:30.714143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:30.714143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:30.714143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:30.714143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:30.714143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:30.714143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:30.714143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:30.714143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:30.714143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:30.714143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:30.714143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:30.714143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:30.714143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:30.715079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:30.715079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:30.715079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:30.715079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 381/741] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:30.715079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fde6e-65f2-4f38-a982-9b65717c4da6 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:30.741776 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb4dea-fba3-4ead-82c0-58c316906694 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:30.778633 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aed427-1138-4c91-89e2-2dcbddf2eab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:30.779235 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aed427-1138-4c91-89e2-2dcbddf2eab2 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:30.779514 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aed427-1138-4c91-89e2-2dcbddf2eab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:30.779784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aed427-1138-4c91-89e2-2dcbddf2eab2 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:30.780441 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aed427-1138-4c91-89e2-2dcbddf2eab2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:30.780739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aed427-1138-4c91-89e2-2dcbddf2eab2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:30.782132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aed427-1138-4c91-89e2-2dcbddf2eab2 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:30.795756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 361/742] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:54:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:30.795756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aed427-1138-4c91-89e2-2dcbddf2eab2 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:30.848303 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fde6e-65f2-4f38-a982-9b65717c4da6 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:30.879592 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:30.900679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aed427-1138-4c91-89e2-2dcbddf2eab2 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:30.904286 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5583b-5d17-47cc-9c5c-2dcf0367a917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:30.905295 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5583b-5d17-47cc-9c5c-2dcf0367a917 None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:30.905775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5583b-5d17-47cc-9c5c-2dcf0367a917 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:30.906295 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5583b-5d17-47cc-9c5c-2dcf0367a917 None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:30.907474 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5583b-5d17-47cc-9c5c-2dcf0367a917 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:30.910593 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5583b-5d17-47cc-9c5c-2dcf0367a917 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:30.944314 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 382/743] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 14 01:54:30 2022] POST /identity/v3/services => generated 251 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:54:30.944314 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5583b-5d17-47cc-9c5c-2dcf0367a917 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:30.975114 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4ef66-12b4-4cef-8f39-33670459a61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:30.976420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4ef66-12b4-4cef-8f39-33670459a61b None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:30.977047 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4ef66-12b4-4cef-8f39-33670459a61b None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:30.977530 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4ef66-12b4-4cef-8f39-33670459a61b None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:30.978866 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d4ef66-12b4-4cef-8f39-33670459a61b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:30.979569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d4ef66-12b4-4cef-8f39-33670459a61b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:30.982549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d4ef66-12b4-4cef-8f39-33670459a61b None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:31.008563 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 362/744] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:31.010244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4ef66-12b4-4cef-8f39-33670459a61b None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:31.012410 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:31.073949 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5583b-5d17-47cc-9c5c-2dcf0367a917 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:31.119959 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383cf0b-6bc5-48b3-87be-7089f10219e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:31.120539 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383cf0b-6bc5-48b3-87be-7089f10219e4 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:31.120889 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383cf0b-6bc5-48b3-87be-7089f10219e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:31.121232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383cf0b-6bc5-48b3-87be-7089f10219e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:31.122014 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383cf0b-6bc5-48b3-87be-7089f10219e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:31.122360 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383cf0b-6bc5-48b3-87be-7089f10219e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:31.123946 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383cf0b-6bc5-48b3-87be-7089f10219e4 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:31.138227 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 383/745] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:31.139360 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383cf0b-6bc5-48b3-87be-7089f10219e4 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:31.156538 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4ef66-12b4-4cef-8f39-33670459a61b None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:31.203227 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf98f0-7916-4630-900b-418545c90c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:31.203804 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf98f0-7916-4630-900b-418545c90c4e None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:31.204085 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf98f0-7916-4630-900b-418545c90c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:31.204357 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf98f0-7916-4630-900b-418545c90c4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:31.212449 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:31.216797 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf98f0-7916-4630-900b-418545c90c4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:31.220088 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf98f0-7916-4630-900b-418545c90c4e None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:31.241051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-3acf98f0-7916-4630-900b-418545c90c4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:31.241051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:31.241051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:31.241051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:31.241051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:31.241051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:31.241051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:31.241051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:31.241051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:31.241051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:31.241051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:31.241051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:31.241051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:31.241051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:31.241051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:31.241051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:31.242757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:31.242757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:31.242757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:31.242757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:31.242757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:31.242757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:31.242757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:31.242757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:31.242757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:31.242757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:31.242757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:31.242757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:31.242757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:31.242757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:31.242757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:31.242757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:31.242757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:31.243671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:31.243671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:31.243671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:31.243671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 363/746] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:31.249040 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf98f0-7916-4630-900b-418545c90c4e None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:31.261065 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383cf0b-6bc5-48b3-87be-7089f10219e4 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:31.280379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:31.302738 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69c3fa-a2c3-4018-8e0f-112ab7183f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:31.303709 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69c3fa-a2c3-4018-8e0f-112ab7183f68 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:31.304230 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69c3fa-a2c3-4018-8e0f-112ab7183f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:31.304698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69c3fa-a2c3-4018-8e0f-112ab7183f68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:31.305613 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd69c3fa-a2c3-4018-8e0f-112ab7183f68 None admin] RBAC: Adding query filter params (role.id=aa420c2fb1ab4bb3b05b004a4d89779a, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=21f481c1f60043e9b0e25ba49b30715f) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:31.306161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd69c3fa-a2c3-4018-8e0f-112ab7183f68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:31.307813 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd69c3fa-a2c3-4018-8e0f-112ab7183f68 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:31.326089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 384/747] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:54:31 2022] GET /identity/v3/role_assignments?user.id=21f481c1f60043e9b0e25ba49b30715f&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=aa420c2fb1ab4bb3b05b004a4d89779a => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:31.327145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69c3fa-a2c3-4018-8e0f-112ab7183f68 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:31.397588 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf98f0-7916-4630-900b-418545c90c4e None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:31.440058 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaedc40-5cb3-4e1b-8a7f-4d302f3904a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:31.440554 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaedc40-5cb3-4e1b-8a7f-4d302f3904a1 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:31.440833 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaedc40-5cb3-4e1b-8a7f-4d302f3904a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:31.441105 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaedc40-5cb3-4e1b-8a7f-4d302f3904a1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:31.449751 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaedc40-5cb3-4e1b-8a7f-4d302f3904a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:31.451284 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaedc40-5cb3-4e1b-8a7f-4d302f3904a1 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:31.464402 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-1eaedc40-5cb3-4e1b-8a7f-4d302f3904a1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:54:31.464402 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:31.464402 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:31.464402 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:31.464402 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:31.464402 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:31.464402 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:31.464402 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:31.464402 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:31.464402 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:31.464402 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:31.464402 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:31.464402 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:31.464402 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:31.464402 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:31.464402 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:31.465412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:31.465412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:31.465412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:31.465412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:31.465412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:31.465412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:31.465412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:31.465412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:31.465412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:31.465412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:31.465412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:31.465412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:31.465412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:31.465412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:31.465412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:31.465412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:31.465412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:31.465412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:31.466841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:31.466841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:31.466841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:31.466841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:31.466841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:31.466841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:31.466841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:31.466841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:31.466841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:54:31.466841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:31.466841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 364/748] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Sep 14 01:54:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:31.471595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fa82f-a700-4057-b666-f39e79f6714c None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:31.471884 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fa82f-a700-4057-b666-f39e79f6714c None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:31.472161 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fa82f-a700-4057-b666-f39e79f6714c None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:31.473934 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 365/749] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:31.475523 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fa82f-a700-4057-b666-f39e79f6714c None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:31.508716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69c3fa-a2c3-4018-8e0f-112ab7183f68 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:31.550698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca2177-cad8-4f4c-a4a4-9ac4b5de82a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:31.551704 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca2177-cad8-4f4c-a4a4-9ac4b5de82a8 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:31.552191 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca2177-cad8-4f4c-a4a4-9ac4b5de82a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:31.552656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca2177-cad8-4f4c-a4a4-9ac4b5de82a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:31.553481 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca2177-cad8-4f4c-a4a4-9ac4b5de82a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:31.554025 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca2177-cad8-4f4c-a4a4-9ac4b5de82a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:31.555582 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca2177-cad8-4f4c-a4a4-9ac4b5de82a8 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:31.562396 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:31.573418 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 385/750] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:31.573418 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cdd0c-ef65-4549-a9ea-4263b15aeacf None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:31.573418 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cdd0c-ef65-4549-a9ea-4263b15aeacf None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:31.573418 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cdd0c-ef65-4549-a9ea-4263b15aeacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:31.594706 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-519cdd0c-ef65-4549-a9ea-4263b15aeacf None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:31.597240 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-519cdd0c-ef65-4549-a9ea-4263b15aeacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:31.601735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fa82f-a700-4057-b666-f39e79f6714c None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:31.641868 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:31.642674 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5635e-9cd7-446c-9a66-d61f075e0df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:31.643512 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5635e-9cd7-446c-9a66-d61f075e0df4 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:31.643625 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5635e-9cd7-446c-9a66-d61f075e0df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:31.643989 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5635e-9cd7-446c-9a66-d61f075e0df4 None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:31.644792 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5635e-9cd7-446c-9a66-d61f075e0df4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:31.645245 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5635e-9cd7-446c-9a66-d61f075e0df4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:31.647452 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5635e-9cd7-446c-9a66-d61f075e0df4 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:31.671552 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 366/751] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Sep 14 01:54:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:54:31.671552 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5635e-9cd7-446c-9a66-d61f075e0df4 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:31.707991 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 386/752] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:54:31.709527 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519cdd0c-ef65-4549-a9ea-4263b15aeacf None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:31.781205 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5635e-9cd7-446c-9a66-d61f075e0df4 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:31.792686 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:31.830079 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39586f6b-2288-472c-a4ef-c96a981e4891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:31.831087 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39586f6b-2288-472c-a4ef-c96a981e4891 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:31.832801 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39586f6b-2288-472c-a4ef-c96a981e4891 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:31.833308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39586f6b-2288-472c-a4ef-c96a981e4891 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:31.849723 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39586f6b-2288-472c-a4ef-c96a981e4891 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:31.851907 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519cdd0c-ef65-4549-a9ea-4263b15aeacf None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:31.853604 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39586f6b-2288-472c-a4ef-c96a981e4891 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:31.882243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-39586f6b-2288-472c-a4ef-c96a981e4891 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:54:31.882243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:31.882243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:31.882243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:31.882243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:31.882243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:31.882243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:31.882243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:31.882243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:31.882243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:31.882243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:31.882243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:31.882243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:31.882243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:31.882243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:31.882243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:31.883470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:31.883470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:31.883470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:31.883470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:31.883470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:31.883470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:31.883470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:31.883470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:31.883470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:31.883470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:31.883470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:31.883470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:31.883470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:31.883470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:31.883470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:31.883470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:31.883470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:31.883470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:31.884680 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:31.884680 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:31.884680 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:31.884680 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:31.884680 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:31.884680 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:31.884680 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:31.884680 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:31.884680 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:54:31.884680 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:31.887009 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 367/753] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Sep 14 01:54:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:31.892419 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39586f6b-2288-472c-a4ef-c96a981e4891 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:31.901000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f2f28-9ed1-4352-8c11-51973c41aab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:31.902718 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f2f28-9ed1-4352-8c11-51973c41aab3 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:31.903106 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f2f28-9ed1-4352-8c11-51973c41aab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:31.903484 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f2f28-9ed1-4352-8c11-51973c41aab3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:31.947291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:32.013885 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f2f28-9ed1-4352-8c11-51973c41aab3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:32.015386 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f2f28-9ed1-4352-8c11-51973c41aab3 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:32.081057 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39586f6b-2288-472c-a4ef-c96a981e4891 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:32.128487 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c706f5-bd65-4554-b084-04893309a89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:32.129122 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c706f5-bd65-4554-b084-04893309a89d None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:32.129403 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c706f5-bd65-4554-b084-04893309a89d None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:32.129674 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c706f5-bd65-4554-b084-04893309a89d None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:32.130183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 387/754] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:54:31 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:54:32.131101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c706f5-bd65-4554-b084-04893309a89d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:32.131283 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f2f28-9ed1-4352-8c11-51973c41aab3 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:32.131442 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c706f5-bd65-4554-b084-04893309a89d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:32.132976 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c706f5-bd65-4554-b084-04893309a89d None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:32.158079 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 368/755] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Sep 14 01:54:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:54:32.158953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c706f5-bd65-4554-b084-04893309a89d None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:32.199101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:32.211464 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:32.259612 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f2f28-9ed1-4352-8c11-51973c41aab3 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:32.283197 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c706f5-bd65-4554-b084-04893309a89d None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:32.304753 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a98b81-cadc-46fb-972b-bdee9161c44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:32.305674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a98b81-cadc-46fb-972b-bdee9161c44e None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:32.306164 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a98b81-cadc-46fb-972b-bdee9161c44e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:32.306612 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a98b81-cadc-46fb-972b-bdee9161c44e None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:32.307509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a98b81-cadc-46fb-972b-bdee9161c44e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:32.309130 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a98b81-cadc-46fb-972b-bdee9161c44e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:32.333208 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d7449-158d-4de4-8b80-a726305fa0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:32.333569 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 388/756] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 14 01:54:32 2022] POST /identity/v3/services => generated 240 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:54:32.333721 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d7449-158d-4de4-8b80-a726305fa0ec None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:32.334075 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d7449-158d-4de4-8b80-a726305fa0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:32.334280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d7449-158d-4de4-8b80-a726305fa0ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:32.335408 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a98b81-cadc-46fb-972b-bdee9161c44e None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:32.350762 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d7449-158d-4de4-8b80-a726305fa0ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:32.350975 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d7449-158d-4de4-8b80-a726305fa0ec None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:32.370051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-1f4d7449-158d-4de4-8b80-a726305fa0ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:32.370051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:32.370051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:32.370051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:32.370051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:32.370051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:32.370051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:32.370051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:32.370051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:32.370051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:32.370051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:32.370051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:32.370051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:32.370051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:32.370051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:32.370051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:32.371232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:32.371232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:32.371232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:32.371232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:32.371232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:32.371232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:32.371232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:32.371232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:32.371232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:32.371232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:32.371232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:32.371232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:32.371232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:32.371232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:32.371232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:32.371232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:32.371232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:32.372659 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:32.372659 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:32.372659 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:32.372659 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 369/757] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:32.376755 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d7449-158d-4de4-8b80-a726305fa0ec None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:32.469009 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:32.499957 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d7449-158d-4de4-8b80-a726305fa0ec None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:32.540262 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228743c-6728-4023-b9ca-e3a61f2da29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:32.541250 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228743c-6728-4023-b9ca-e3a61f2da29d None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:32.541746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228743c-6728-4023-b9ca-e3a61f2da29d None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:32.542381 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228743c-6728-4023-b9ca-e3a61f2da29d None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:32.543251 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228743c-6728-4023-b9ca-e3a61f2da29d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:32.543747 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228743c-6728-4023-b9ca-e3a61f2da29d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:32.545670 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228743c-6728-4023-b9ca-e3a61f2da29d None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:32.560425 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 370/758] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:32.566256 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228743c-6728-4023-b9ca-e3a61f2da29d None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:32.602927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a98b81-cadc-46fb-972b-bdee9161c44e None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:32.634043 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:32.644142 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df73f13-7cfa-467c-b256-6dde9e8b04ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:32.644731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df73f13-7cfa-467c-b256-6dde9e8b04ac None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:32.645896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df73f13-7cfa-467c-b256-6dde9e8b04ac None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:32.646511 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df73f13-7cfa-467c-b256-6dde9e8b04ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:32.658445 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df73f13-7cfa-467c-b256-6dde9e8b04ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:32.659990 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df73f13-7cfa-467c-b256-6dde9e8b04ac None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:32.676393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-7df73f13-7cfa-467c-b256-6dde9e8b04ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:32.676393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:32.676393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:32.676393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:32.676393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:32.676393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:32.676393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:32.676393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:32.676393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:32.676393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:32.676393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:32.676393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:32.676393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:32.676393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:32.676393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:32.676393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:32.677427 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:32.677427 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:32.677427 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:32.677427 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:32.677427 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:32.677427 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:32.677427 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:32.677427 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:32.677427 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:32.677427 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:32.677427 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:32.677427 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:32.677427 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:32.677427 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:32.677427 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:32.677427 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:32.677427 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:32.678365 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:32.678365 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:32.678365 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:32.682272 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 389/759] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:32.684512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df73f13-7cfa-467c-b256-6dde9e8b04ac None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:32.698249 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228743c-6728-4023-b9ca-e3a61f2da29d None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:32.746934 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4faa343-0e36-42dd-8fe1-c4843877dddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:32.747673 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4faa343-0e36-42dd-8fe1-c4843877dddd None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:32.748020 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4faa343-0e36-42dd-8fe1-c4843877dddd None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:32.748399 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4faa343-0e36-42dd-8fe1-c4843877dddd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:32.756463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4faa343-0e36-42dd-8fe1-c4843877dddd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:32.758074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4faa343-0e36-42dd-8fe1-c4843877dddd None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:32.770544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-f4faa343-0e36-42dd-8fe1-c4843877dddd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:32.770544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:32.770544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:32.770544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:32.770544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:32.770544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:32.770544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:32.770544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:32.770544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:32.770544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:32.770544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:32.770544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:32.770544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:32.770544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:32.770544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:32.771497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:32.771497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:32.771497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:32.771497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:32.771497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:32.771497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:32.771497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:32.771497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:32.771497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:32.771497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:32.771497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:32.771497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:32.771497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:32.771497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:32.771497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:32.771497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:32.771497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:32.772442 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:32.772442 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:32.772442 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:32.772442 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:32.772442 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:32.772442 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:32.772442 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:32.772442 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:32.772442 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 371/760] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:32.774309 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:32.778761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4faa343-0e36-42dd-8fe1-c4843877dddd None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:32.819289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df73f13-7cfa-467c-b256-6dde9e8b04ac None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:32.889000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f194e96-0050-400f-9c75-2c73bfee0d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:32.891073 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f194e96-0050-400f-9c75-2c73bfee0d58 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:32.891523 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f194e96-0050-400f-9c75-2c73bfee0d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:32.893880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f194e96-0050-400f-9c75-2c73bfee0d58 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:32.894691 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f194e96-0050-400f-9c75-2c73bfee0d58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:32.895140 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f194e96-0050-400f-9c75-2c73bfee0d58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:32.898102 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f194e96-0050-400f-9c75-2c73bfee0d58 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:32.928151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 390/761] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:32.934212 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f194e96-0050-400f-9c75-2c73bfee0d58 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:33.018607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:33.068250 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4faa343-0e36-42dd-8fe1-c4843877dddd None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:33.108237 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7370500-761a-4ce3-8fb0-2c889b4b4578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:33.108846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7370500-761a-4ce3-8fb0-2c889b4b4578 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:33.109175 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7370500-761a-4ce3-8fb0-2c889b4b4578 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:33.109439 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7370500-761a-4ce3-8fb0-2c889b4b4578 None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:33.111260 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7370500-761a-4ce3-8fb0-2c889b4b4578 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:33.111566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7370500-761a-4ce3-8fb0-2c889b4b4578 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:33.112998 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7370500-761a-4ce3-8fb0-2c889b4b4578 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:33.126707 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 372/762] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:33.139074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7370500-761a-4ce3-8fb0-2c889b4b4578 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:33.139216 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f194e96-0050-400f-9c75-2c73bfee0d58 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:33.188279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea13ae8-c2cf-4979-98a0-939b4e8d2d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:33.189004 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea13ae8-c2cf-4979-98a0-939b4e8d2d05 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:33.189284 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea13ae8-c2cf-4979-98a0-939b4e8d2d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:33.189553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea13ae8-c2cf-4979-98a0-939b4e8d2d05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:33.200491 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea13ae8-c2cf-4979-98a0-939b4e8d2d05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:33.201960 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea13ae8-c2cf-4979-98a0-939b4e8d2d05 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:33.214502 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:33.232894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-5ea13ae8-c2cf-4979-98a0-939b4e8d2d05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:33.232894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:33.232894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:33.232894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:33.232894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:33.232894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:33.232894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:33.232894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:33.232894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:33.232894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:33.232894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:33.232894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:33.232894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:33.232894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:33.232894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:33.233849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:33.233849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:33.233849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:33.233849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:33.233849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:33.233849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:33.233849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:33.233849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:33.233849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:33.233849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:33.233849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:33.233849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:33.233849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:33.233849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:33.233849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:33.233849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:33.233849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:33.234799 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:33.234799 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:33.234799 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:33.234799 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:33.234799 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:33.234799 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:33.234799 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:33.234799 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:33.234799 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 391/763] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:33.250215 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea13ae8-c2cf-4979-98a0-939b4e8d2d05 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:33.269444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7370500-761a-4ce3-8fb0-2c889b4b4578 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:33.305566 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38db8d-a0f9-4a91-b31a-b729e4d9c036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:33.306197 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38db8d-a0f9-4a91-b31a-b729e4d9c036 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:33.306470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38db8d-a0f9-4a91-b31a-b729e4d9c036 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:33.306737 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38db8d-a0f9-4a91-b31a-b729e4d9c036 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:33.307456 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38db8d-a0f9-4a91-b31a-b729e4d9c036 None admin] RBAC: Adding query filter params (role.id=e962a2a280224f9fbcd0fdc02262993d, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=e1c8d2eec886482eb135e19ded2c61ae) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:33.307813 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38db8d-a0f9-4a91-b31a-b729e4d9c036 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:33.309293 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38db8d-a0f9-4a91-b31a-b729e4d9c036 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:33.315923 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 373/764] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:54:33 2022] GET /identity/v3/role_assignments?user.id=e1c8d2eec886482eb135e19ded2c61ae&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=e962a2a280224f9fbcd0fdc02262993d => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:33.317222 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d450a-a2d8-45e4-b21b-87c96a31f8b0 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:33.317504 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d450a-a2d8-45e4-b21b-87c96a31f8b0 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:33.317777 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d450a-a2d8-45e4-b21b-87c96a31f8b0 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:33.319410 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 374/765] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:33.326893 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ecffa-9bb4-441a-a653-32489eb2ff38 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:33.327270 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ecffa-9bb4-441a-a653-32489eb2ff38 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:33.327613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ecffa-9bb4-441a-a653-32489eb2ff38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:33.367193 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-3f0ecffa-9bb4-441a-a653-32489eb2ff38 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:33.368432 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-3f0ecffa-9bb4-441a-a653-32489eb2ff38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:33.406978 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:33.410557 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea13ae8-c2cf-4979-98a0-939b4e8d2d05 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:33.451127 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143050b-1bda-4de5-a50d-7b347443443e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:33.451762 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143050b-1bda-4de5-a50d-7b347443443e None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:33.452043 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143050b-1bda-4de5-a50d-7b347443443e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:33.452299 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143050b-1bda-4de5-a50d-7b347443443e None admin] PATH_INFO: `/v3/projects` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:33.453121 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143050b-1bda-4de5-a50d-7b347443443e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:33.453432 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143050b-1bda-4de5-a50d-7b347443443e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:33.455547 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143050b-1bda-4de5-a50d-7b347443443e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:33.485050 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 392/766] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:33.494301 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143050b-1bda-4de5-a50d-7b347443443e None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:33.514084 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 375/767] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:54:33.609208 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143050b-1bda-4de5-a50d-7b347443443e None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:33.649589 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90670c-b18b-420c-976d-6e7037d4d15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:33.650270 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90670c-b18b-420c-976d-6e7037d4d15e None admin] REQUEST_METHOD: `PUT` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:33.650545 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90670c-b18b-420c-976d-6e7037d4d15e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:33.650811 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90670c-b18b-420c-976d-6e7037d4d15e None admin] PATH_INFO: `/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/f721a3a4eca941989685448a87bb9a19/roles/aa420c2fb1ab4bb3b05b004a4d89779a` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:33.655265 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90670c-b18b-420c-976d-6e7037d4d15e None admin] RBAC: Authorizing `identity:create_grant(project_id=9e04b16dba1c4de79dc6b1a5e16ed907, user_id=f721a3a4eca941989685448a87bb9a19, role_id=aa420c2fb1ab4bb3b05b004a4d89779a)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:33.656776 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90670c-b18b-420c-976d-6e7037d4d15e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:33.673708 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 393/768] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:54:33 2022] PUT /identity/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/f721a3a4eca941989685448a87bb9a19/roles/aa420c2fb1ab4bb3b05b004a4d89779a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:54:33.822014 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ecffa-9bb4-441a-a653-32489eb2ff38 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:33.836962 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba8404-79b2-4293-9194-e3d128c49a54 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:33.837659 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba8404-79b2-4293-9194-e3d128c49a54 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:33.837659 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba8404-79b2-4293-9194-e3d128c49a54 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:33.842082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 394/769] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:54:33.844882 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0665f9c-ad69-4b59-806f-e29f5f8042dd None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:33.846678 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0665f9c-ad69-4b59-806f-e29f5f8042dd None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:33.846961 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0665f9c-ad69-4b59-806f-e29f5f8042dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:33.879814 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-f0665f9c-ad69-4b59-806f-e29f5f8042dd None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:33.881305 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-f0665f9c-ad69-4b59-806f-e29f5f8042dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:33.908522 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:33.914607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:33.989419 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ecffa-9bb4-441a-a653-32489eb2ff38 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:33.994332 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 395/770] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:54:34.030052 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17256232-3617-481b-b54d-02b664530680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:34.030691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17256232-3617-481b-b54d-02b664530680 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:34.030960 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17256232-3617-481b-b54d-02b664530680 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:34.031223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17256232-3617-481b-b54d-02b664530680 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:34.031800 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17256232-3617-481b-b54d-02b664530680 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:34.033184 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17256232-3617-481b-b54d-02b664530680 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:34.042231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-17256232-3617-481b-b54d-02b664530680 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 01:54:34.042231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:34.042231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:54:34.042231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:54:34.042231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:54:34.042231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:54:34.042231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:54:34.042231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:34.042231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 01:54:34.042231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:34.042231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:54:34.042231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:34.042231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:34.042231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:34.042231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:34.042231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:34.042231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:34.044458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:34.044458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:34.044458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:34.044458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:34.044458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:34.044458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:34.044458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:54:34.044458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:54:34.044458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:54:34.044458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:54:34.044458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:34.044458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:34.044458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:34.044458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:34.044458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:34.044458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:34.045679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:34.045679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:34.045679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:34.045679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:34.045679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:34.045679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:34.045679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:34.045679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:34.045679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:34.045679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:34.045679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:54:34.045679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:34.045679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 01:54:34.045679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:34.045679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 376/771] 10.222.0.15 () {56 vars in 1129 bytes} [Wed Sep 14 01:54:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:34.049148 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0665f9c-ad69-4b59-806f-e29f5f8042dd None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:34.182495 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0665f9c-ad69-4b59-806f-e29f5f8042dd None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:34.235615 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2600a77-a0ce-4a37-8656-37ae2f129ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:34.236555 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2600a77-a0ce-4a37-8656-37ae2f129ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:34.237008 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2600a77-a0ce-4a37-8656-37ae2f129ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:34.237477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2600a77-a0ce-4a37-8656-37ae2f129ea5 None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:34.238300 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2600a77-a0ce-4a37-8656-37ae2f129ea5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:34.238776 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2600a77-a0ce-4a37-8656-37ae2f129ea5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:34.240372 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2600a77-a0ce-4a37-8656-37ae2f129ea5 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:34.249689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 396/772] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Sep 14 01:54:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:34.256142 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17256232-3617-481b-b54d-02b664530680 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:34.378890 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2600a77-a0ce-4a37-8656-37ae2f129ea5 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:34.412082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:34.453794 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17256232-3617-481b-b54d-02b664530680 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:34.476243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:34.512208 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7907be-81e6-48d3-b647-32b790e2801a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:34.512743 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7907be-81e6-48d3-b647-32b790e2801a None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:34.513019 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7907be-81e6-48d3-b647-32b790e2801a None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:34.513291 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7907be-81e6-48d3-b647-32b790e2801a None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:34.513947 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7907be-81e6-48d3-b647-32b790e2801a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:34.514263 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7907be-81e6-48d3-b647-32b790e2801a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:34.515661 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7907be-81e6-48d3-b647-32b790e2801a None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:34.521966 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 377/773] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Sep 14 01:54:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:34.527178 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7907be-81e6-48d3-b647-32b790e2801a None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:34.540418 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2600a77-a0ce-4a37-8656-37ae2f129ea5 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:34.577339 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f6047-4ad2-4916-8dd4-b227a5360e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:34.578437 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f6047-4ad2-4916-8dd4-b227a5360e0a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:34.578916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f6047-4ad2-4916-8dd4-b227a5360e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:34.579910 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f6047-4ad2-4916-8dd4-b227a5360e0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:34.591185 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f6047-4ad2-4916-8dd4-b227a5360e0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:34.596639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-488f6047-4ad2-4916-8dd4-b227a5360e0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:54:34.596639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:34.596639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:34.596639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:34.596639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:34.596639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:34.596639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:34.596639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:34.596639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:34.596639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:34.596639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:34.596639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:34.596639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:34.596639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:34.596639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:34.596639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:34.597567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:34.597567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:34.597567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:34.597567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:34.597567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:34.597567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:34.597567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:34.597567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:34.597567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:34.597567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:34.597567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:34.597567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:34.597567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:34.597567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:34.597567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:34.597567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:34.598809 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:34.598809 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:34.598809 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:34.598809 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:34.598809 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:34.598809 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:34.598809 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:34.598809 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:34.598809 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:34.598809 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:34.598809 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:54:34.598809 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:34.601266 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 397/774] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:54:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:34.607321 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f6047-4ad2-4916-8dd4-b227a5360e0a None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:34.654323 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7907be-81e6-48d3-b647-32b790e2801a None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:34.720602 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:34.721856 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53bc17-5180-4c99-a1a9-2ae1c4fb19d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:34.722742 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53bc17-5180-4c99-a1a9-2ae1c4fb19d8 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:34.723287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53bc17-5180-4c99-a1a9-2ae1c4fb19d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:34.723752 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53bc17-5180-4c99-a1a9-2ae1c4fb19d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:34.724799 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53bc17-5180-4c99-a1a9-2ae1c4fb19d8 None admin] RBAC: Adding query filter params (interface=public, service_id=9c5eb2bae5ea4bcda57865ce73714e32, region_id=RegionOne) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:34.727155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53bc17-5180-4c99-a1a9-2ae1c4fb19d8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:34.728848 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53bc17-5180-4c99-a1a9-2ae1c4fb19d8 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:34.748220 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 378/775] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Sep 14 01:54:34 2022] GET /identity/v3/endpoints?service_id=9c5eb2bae5ea4bcda57865ce73714e32&interface=public®ion_id=RegionOne => generated 194 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:34.754413 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53bc17-5180-4c99-a1a9-2ae1c4fb19d8 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:34.778203 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f6047-4ad2-4916-8dd4-b227a5360e0a None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:34.822364 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:34.844722 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357beb4e-e460-4cee-89ad-f52a45678e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:34.845904 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357beb4e-e460-4cee-89ad-f52a45678e2d None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:34.846163 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357beb4e-e460-4cee-89ad-f52a45678e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:34.846434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357beb4e-e460-4cee-89ad-f52a45678e2d None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:34.847092 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357beb4e-e460-4cee-89ad-f52a45678e2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:34.847384 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357beb4e-e460-4cee-89ad-f52a45678e2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:34.854082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357beb4e-e460-4cee-89ad-f52a45678e2d None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:34.873849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 398/776] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:54:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:34.878681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357beb4e-e460-4cee-89ad-f52a45678e2d None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:34.934374 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53bc17-5180-4c99-a1a9-2ae1c4fb19d8 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:35.009148 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92b937-0e69-41f0-a3bb-de8ba6b8590e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:35.009807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92b937-0e69-41f0-a3bb-de8ba6b8590e None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:35.010115 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92b937-0e69-41f0-a3bb-de8ba6b8590e None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:35.017285 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92b937-0e69-41f0-a3bb-de8ba6b8590e None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:35.017957 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92b937-0e69-41f0-a3bb-de8ba6b8590e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:35.019519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92b937-0e69-41f0-a3bb-de8ba6b8590e None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:35.025667 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 379/777] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Sep 14 01:54:34 2022] GET /identity/v3/services => generated 772 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:35.065963 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357beb4e-e460-4cee-89ad-f52a45678e2d None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:35.117774 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45056c4e-d3a8-4446-ac02-eafa5e0da2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:35.118518 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45056c4e-d3a8-4446-ac02-eafa5e0da2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:35.118797 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45056c4e-d3a8-4446-ac02-eafa5e0da2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:35.119070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45056c4e-d3a8-4446-ac02-eafa5e0da2b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:35.124973 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:35.138498 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45056c4e-d3a8-4446-ac02-eafa5e0da2b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:35.140070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45056c4e-d3a8-4446-ac02-eafa5e0da2b9 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:35.156594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-45056c4e-d3a8-4446-ac02-eafa5e0da2b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:35.156594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:35.156594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:35.156594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:35.156594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:35.156594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:35.156594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:35.156594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:35.156594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:35.156594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:35.156594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:35.156594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:35.156594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:35.156594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:35.156594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:35.156594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:35.157730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:35.157730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:35.157730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:35.157730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:35.157730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:35.157730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:35.157730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:35.157730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:35.157730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:35.157730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:35.157730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:35.157730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:35.157730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:35.157730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:35.157730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:35.157730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:35.157730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:35.158802 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:35.158802 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:35.158802 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:35.158802 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 399/778] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:35.165453 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92b937-0e69-41f0-a3bb-de8ba6b8590e None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:35.258889 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e98eb-0830-4cd3-b647-fc5660768e94 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:35.259473 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e98eb-0830-4cd3-b647-fc5660768e94 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:35.259747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e98eb-0830-4cd3-b647-fc5660768e94 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:35.261201 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 400/779] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:35.266301 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7efc7-dfb7-4200-8cff-577b28a4ada5 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:35.266577 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7efc7-dfb7-4200-8cff-577b28a4ada5 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:35.267365 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7efc7-dfb7-4200-8cff-577b28a4ada5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:35.290275 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-cac7efc7-dfb7-4200-8cff-577b28a4ada5 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:35.291450 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-cac7efc7-dfb7-4200-8cff-577b28a4ada5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:35.291911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92b937-0e69-41f0-a3bb-de8ba6b8590e None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:35.294849 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:35.331068 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0337d4-a7f8-4c77-8507-05943ed6b091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:35.331899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0337d4-a7f8-4c77-8507-05943ed6b091 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:35.332312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0337d4-a7f8-4c77-8507-05943ed6b091 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:35.332695 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0337d4-a7f8-4c77-8507-05943ed6b091 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:35.333342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0337d4-a7f8-4c77-8507-05943ed6b091 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:35.333638 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0337d4-a7f8-4c77-8507-05943ed6b091 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:35.335326 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0337d4-a7f8-4c77-8507-05943ed6b091 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:35.349651 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 380/780] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:35.355741 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0337d4-a7f8-4c77-8507-05943ed6b091 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:35.377731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 401/781] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:54:35.514406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0337d4-a7f8-4c77-8507-05943ed6b091 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:35.548962 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f6904-7469-44b6-bab2-5438478808fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:35.549590 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f6904-7469-44b6-bab2-5438478808fd None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:35.549860 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f6904-7469-44b6-bab2-5438478808fd None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:35.550157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f6904-7469-44b6-bab2-5438478808fd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:35.557621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f6904-7469-44b6-bab2-5438478808fd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:35.559074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f6904-7469-44b6-bab2-5438478808fd None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:35.570763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-b49f6904-7469-44b6-bab2-5438478808fd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:54:35.570763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:35.570763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:35.570763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:35.570763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:35.570763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:35.570763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:35.570763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:35.570763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:35.570763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:35.570763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:35.570763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:35.570763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:35.570763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:35.570763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:35.570763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:35.571754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:35.571754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:35.571754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:35.571754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:35.571754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:35.571754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:35.571754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:35.571754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:35.571754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:35.571754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:35.571754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:35.571754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:35.571754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:35.571754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:35.571754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:35.571754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:35.571754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:35.571754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:35.573113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:35.573113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:35.573113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:35.573113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:35.573113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:35.573113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:35.573113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:35.573113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:35.573113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:54:35.573113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:35.573113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 381/782] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:54:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:35.577469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7efc7-dfb7-4200-8cff-577b28a4ada5 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:35.684127 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f6904-7469-44b6-bab2-5438478808fd None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:35.717516 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7efc7-dfb7-4200-8cff-577b28a4ada5 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:35.754339 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:35.756092 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:35.759671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6a6c0-2879-4621-962a-4fd35db21b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:35.760283 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6a6c0-2879-4621-962a-4fd35db21b67 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:35.760561 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6a6c0-2879-4621-962a-4fd35db21b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:35.760828 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6a6c0-2879-4621-962a-4fd35db21b67 None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:35.761607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a6a6c0-2879-4621-962a-4fd35db21b67 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:35.761915 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a6a6c0-2879-4621-962a-4fd35db21b67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:35.763391 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a6a6c0-2879-4621-962a-4fd35db21b67 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:35.804511 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 402/783] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Sep 14 01:54:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:35.812095 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6a6c0-2879-4621-962a-4fd35db21b67 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:35.871634 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f6904-7469-44b6-bab2-5438478808fd None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:35.938538 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247387d-e56e-4703-9cea-051a4fd61141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:35.939684 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247387d-e56e-4703-9cea-051a4fd61141 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:35.939962 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247387d-e56e-4703-9cea-051a4fd61141 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:35.940227 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247387d-e56e-4703-9cea-051a4fd61141 None admin] PATH_INFO: `/v3/services/network` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:35.940813 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247387d-e56e-4703-9cea-051a4fd61141 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:35.942205 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247387d-e56e-4703-9cea-051a4fd61141 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:35.960157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-a247387d-e56e-4703-9cea-051a4fd61141 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 01:54:35.960157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:35.960157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:54:35.960157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:54:35.960157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:54:35.960157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:54:35.960157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:54:35.960157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:35.960157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 01:54:35.960157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:35.960157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:54:35.960157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:35.960157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:35.960157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:35.960157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:35.960157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:35.960157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:35.961241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:35.961241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:35.961241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:35.961241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:35.961241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:35.961241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:35.961241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:54:35.961241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:54:35.961241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:54:35.961241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:54:35.961241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:35.961241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:35.961241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:35.961241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:35.961241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:35.961241 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:35.962158 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:35.962158 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:35.962158 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:35.962158 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:35.962158 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:35.962158 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:35.962158 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:35.962158 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:35.962158 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:35.962158 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:35.962158 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:54:35.962158 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:35.962158 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 01:54:35.962158 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:35.963476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 382/784] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:35 2022] GET /identity/v3/services/network => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:35.967025 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247387d-e56e-4703-9cea-051a4fd61141 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:36.000986 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6a6c0-2879-4621-962a-4fd35db21b67 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:36.040643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e1aa3-330f-4b29-a766-ae810c46457a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:36.041550 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e1aa3-330f-4b29-a766-ae810c46457a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:36.041839 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e1aa3-330f-4b29-a766-ae810c46457a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:36.042139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e1aa3-330f-4b29-a766-ae810c46457a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:36.055890 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e1aa3-330f-4b29-a766-ae810c46457a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:36.057360 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e1aa3-330f-4b29-a766-ae810c46457a None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:36.072424 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:36.080296 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-488e1aa3-330f-4b29-a766-ae810c46457a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:36.080296 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:36.080296 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:36.080296 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:36.080296 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:36.080296 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:36.080296 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:36.080296 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:36.080296 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:36.080296 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:36.080296 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:36.080296 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:36.080296 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:36.080296 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:36.080296 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:36.080296 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:36.081289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:36.081289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:36.081289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:36.081289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:36.081289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:36.081289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:36.081289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:36.081289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:36.081289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:36.081289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:36.081289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:36.081289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:36.081289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:36.081289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:36.081289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:36.081289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:36.081289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:36.082253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:36.082253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:36.082253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:36.086156 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 403/785] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:36.087454 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed104fb3-cae8-4d38-bbf6-b274af198098 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:36.087739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed104fb3-cae8-4d38-bbf6-b274af198098 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:36.088039 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed104fb3-cae8-4d38-bbf6-b274af198098 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:36.089446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 404/786] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:36.091096 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed104fb3-cae8-4d38-bbf6-b274af198098 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:36.096545 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247387d-e56e-4703-9cea-051a4fd61141 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:36.129776 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:36.143458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ac92e-2da8-4e87-b510-d167249e7a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:36.144069 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ac92e-2da8-4e87-b510-d167249e7a83 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:36.144350 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ac92e-2da8-4e87-b510-d167249e7a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:36.144623 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ac92e-2da8-4e87-b510-d167249e7a83 None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:36.145249 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ac92e-2da8-4e87-b510-d167249e7a83 None admin] RBAC: Adding query filter params (name=network) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:36.145551 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ac92e-2da8-4e87-b510-d167249e7a83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:36.146971 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ac92e-2da8-4e87-b510-d167249e7a83 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:36.152079 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 383/787] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 14 01:54:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:36.153318 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1bcb7-c692-4b43-b0b4-b2878699aee7 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:36.153607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1bcb7-c692-4b43-b0b4-b2878699aee7 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:36.153881 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1bcb7-c692-4b43-b0b4-b2878699aee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:36.173438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-11f1bcb7-c692-4b43-b0b4-b2878699aee7 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:36.174713 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-11f1bcb7-c692-4b43-b0b4-b2878699aee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:36.247545 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed104fb3-cae8-4d38-bbf6-b274af198098 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:36.254367 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 384/788] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:54:36.255646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1bcb7-c692-4b43-b0b4-b2878699aee7 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:36.293105 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ffe15-9cbd-4a33-af6d-57810b370bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:36.294100 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ffe15-9cbd-4a33-af6d-57810b370bcc None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:36.294606 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ffe15-9cbd-4a33-af6d-57810b370bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:36.295812 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ffe15-9cbd-4a33-af6d-57810b370bcc None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:36.296959 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ffe15-9cbd-4a33-af6d-57810b370bcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:36.297480 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ffe15-9cbd-4a33-af6d-57810b370bcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:36.299213 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ffe15-9cbd-4a33-af6d-57810b370bcc None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:36.314589 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 405/789] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:36.323677 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ffe15-9cbd-4a33-af6d-57810b370bcc None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:36.401075 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:36.427126 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1bcb7-c692-4b43-b0b4-b2878699aee7 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:36.429360 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:36.443896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ffe15-9cbd-4a33-af6d-57810b370bcc None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:36.480692 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17df38c-fc2d-452f-a8f9-1c16ced9d921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:36.481296 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17df38c-fc2d-452f-a8f9-1c16ced9d921 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:36.481570 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17df38c-fc2d-452f-a8f9-1c16ced9d921 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:36.481833 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17df38c-fc2d-452f-a8f9-1c16ced9d921 None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:36.482480 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17df38c-fc2d-452f-a8f9-1c16ced9d921 None admin] RBAC: Adding query filter params (type=network) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:36.482775 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17df38c-fc2d-452f-a8f9-1c16ced9d921 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:36.484120 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17df38c-fc2d-452f-a8f9-1c16ced9d921 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:36.485139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7accc6-03f2-44d1-867b-9dbad0530d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:36.486448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7accc6-03f2-44d1-867b-9dbad0530d96 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:36.486822 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7accc6-03f2-44d1-867b-9dbad0530d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:36.487188 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7accc6-03f2-44d1-867b-9dbad0530d96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:36.490143 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 385/790] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 14 01:54:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:36.491293 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6307dd2-81b5-44a9-acc8-53af29c391c5 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:36.491568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6307dd2-81b5-44a9-acc8-53af29c391c5 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:36.491831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6307dd2-81b5-44a9-acc8-53af29c391c5 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:36.493143 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 386/791] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:36.498383 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5d0d3-a17d-42da-87ab-7cc98405f1a9 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:36.498598 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5d0d3-a17d-42da-87ab-7cc98405f1a9 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:36.498867 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5d0d3-a17d-42da-87ab-7cc98405f1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:36.498971 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7accc6-03f2-44d1-867b-9dbad0530d96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:36.500397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7accc6-03f2-44d1-867b-9dbad0530d96 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:36.512244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-ad7accc6-03f2-44d1-867b-9dbad0530d96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:36.512244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:36.512244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:36.512244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:36.512244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:36.512244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:36.512244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:36.512244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:36.512244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:36.512244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:36.512244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:36.512244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:36.512244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:36.512244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:36.512244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:36.513170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:36.513170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:36.513170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:36.513170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:36.513170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:36.513170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:36.513170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:36.513170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:36.513170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:36.513170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:36.513170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:36.513170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:36.513170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:36.513170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:36.513170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:36.513170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:36.513170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:36.514206 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:36.514206 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:36.514206 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:36.514206 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:36.514206 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:36.514206 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:36.514206 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:36.514206 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:36.514206 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 406/792] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:36.515240 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7accc6-03f2-44d1-867b-9dbad0530d96 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:36.522112 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-3db5d0d3-a17d-42da-87ab-7cc98405f1a9 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:36.523387 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-3db5d0d3-a17d-42da-87ab-7cc98405f1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:36.595819 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 387/793] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:54:36.596657 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5d0d3-a17d-42da-87ab-7cc98405f1a9 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:36.624897 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7accc6-03f2-44d1-867b-9dbad0530d96 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:36.675013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918e193-9013-457e-9c17-fee5a917eb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:36.676433 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918e193-9013-457e-9c17-fee5a917eb7c None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:36.676935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918e193-9013-457e-9c17-fee5a917eb7c None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:36.677393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918e193-9013-457e-9c17-fee5a917eb7c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:36.678573 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:36.679801 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5918e193-9013-457e-9c17-fee5a917eb7c None admin] RBAC: Adding query filter params (interface=public, service_id=2fbfdb73f5434680920e41768980bfc5, region_id=RegionOne) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:36.680681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5918e193-9013-457e-9c17-fee5a917eb7c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:36.683274 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5918e193-9013-457e-9c17-fee5a917eb7c None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:36.695113 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 407/794] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Sep 14 01:54:36 2022] GET /identity/v3/endpoints?service_id=2fbfdb73f5434680920e41768980bfc5&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:36.696521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918e193-9013-457e-9c17-fee5a917eb7c None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:36.703549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5d0d3-a17d-42da-87ab-7cc98405f1a9 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:36.717653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:36.750169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb5dda-3a7e-44c4-b558-fd78215bab4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:36.750699 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb5dda-3a7e-44c4-b558-fd78215bab4c None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:36.750972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb5dda-3a7e-44c4-b558-fd78215bab4c None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:36.751784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb5dda-3a7e-44c4-b558-fd78215bab4c None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:36.752852 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb5dda-3a7e-44c4-b558-fd78215bab4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:36.752852 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb5dda-3a7e-44c4-b558-fd78215bab4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:36.754186 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb5dda-3a7e-44c4-b558-fd78215bab4c None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:36.771287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 388/795] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:36.772146 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb5dda-3a7e-44c4-b558-fd78215bab4c None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:36.828543 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918e193-9013-457e-9c17-fee5a917eb7c None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:36.868760 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ed15d-9bf1-4e12-9c8a-d51929b05fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:36.869729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ed15d-9bf1-4e12-9c8a-d51929b05fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:36.870037 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ed15d-9bf1-4e12-9c8a-d51929b05fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:36.870605 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ed15d-9bf1-4e12-9c8a-d51929b05fb1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:36.883232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ed15d-9bf1-4e12-9c8a-d51929b05fb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:36.890367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-7f0ed15d-9bf1-4e12-9c8a-d51929b05fb1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:36.890367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:36.890367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:36.890367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:36.890367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:36.890367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:36.890367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:36.890367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:36.890367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:36.890367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:36.890367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:36.890367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:36.890367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:36.890367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:36.890367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:36.890367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:36.891335 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:36.891335 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:36.891335 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:36.891335 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:36.891335 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:36.891335 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:36.891335 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:36.891335 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:36.891335 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:36.891335 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:36.891335 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:36.891335 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:36.891335 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:36.891335 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:36.891335 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:36.891335 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:36.892766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:36.892766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:36.892766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:36.892766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:36.892766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:36.892766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:36.892766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:36.892766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:36.892766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:36.892766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:36.892766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:36.892766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:36.893416 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb5dda-3a7e-44c4-b558-fd78215bab4c None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:36.894251 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 408/796] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:54:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:36.895086 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ed15d-9bf1-4e12-9c8a-d51929b05fb1 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:36.931374 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e8767-e2a0-4d85-b56f-dcb9dd16e290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:36.932312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e8767-e2a0-4d85-b56f-dcb9dd16e290 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:36.932736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e8767-e2a0-4d85-b56f-dcb9dd16e290 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:36.933154 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e8767-e2a0-4d85-b56f-dcb9dd16e290 None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:36.933895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e8767-e2a0-4d85-b56f-dcb9dd16e290 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:36.935473 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e8767-e2a0-4d85-b56f-dcb9dd16e290 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:36.938419 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:36.942946 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 389/797] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Sep 14 01:54:36 2022] GET /identity/v3/services => generated 772 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:36.944531 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e8767-e2a0-4d85-b56f-dcb9dd16e290 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:36.982941 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:37.027726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ed15d-9bf1-4e12-9c8a-d51929b05fb1 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:37.078011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af6697-e412-4bf8-afdb-a7021d0db0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:37.079306 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af6697-e412-4bf8-afdb-a7021d0db0cb None admin] REQUEST_METHOD: `PUT` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:37.080186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af6697-e412-4bf8-afdb-a7021d0db0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:37.080616 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af6697-e412-4bf8-afdb-a7021d0db0cb None admin] PATH_INFO: `/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/21f481c1f60043e9b0e25ba49b30715f/roles/aa420c2fb1ab4bb3b05b004a4d89779a` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:37.083990 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af6697-e412-4bf8-afdb-a7021d0db0cb None admin] RBAC: Authorizing `identity:create_grant(project_id=9e04b16dba1c4de79dc6b1a5e16ed907, user_id=21f481c1f60043e9b0e25ba49b30715f, role_id=aa420c2fb1ab4bb3b05b004a4d89779a)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:37.086103 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af6697-e412-4bf8-afdb-a7021d0db0cb None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:37.086489 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e8767-e2a0-4d85-b56f-dcb9dd16e290 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:37.107196 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 409/798] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:54:36 2022] PUT /identity/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/21f481c1f60043e9b0e25ba49b30715f/roles/aa420c2fb1ab4bb3b05b004a4d89779a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:54:37.108378 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af6697-e412-4bf8-afdb-a7021d0db0cb None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:37.142286 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4733c9f-07e3-4cf9-9680-017c6db4ef30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:37.142948 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4733c9f-07e3-4cf9-9680-017c6db4ef30 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:37.143228 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4733c9f-07e3-4cf9-9680-017c6db4ef30 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:37.143547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4733c9f-07e3-4cf9-9680-017c6db4ef30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:37.160894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4733c9f-07e3-4cf9-9680-017c6db4ef30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:37.169382 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-a4733c9f-07e3-4cf9-9680-017c6db4ef30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:54:37.169382 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:37.169382 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:37.169382 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:37.169382 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:37.169382 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:37.169382 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:37.169382 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:37.169382 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:37.169382 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:37.169382 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:37.169382 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:37.169382 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:37.169382 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:37.169382 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:37.169382 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:37.170660 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:37.170660 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:37.170660 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:37.170660 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:37.170660 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:37.170660 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:37.170660 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:37.170660 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:37.170660 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:37.170660 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:37.170660 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:37.170660 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:37.170660 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:37.170660 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:37.170660 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:37.170660 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:37.176010 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:37.176010 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:37.176010 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:37.176010 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:37.176010 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:37.176010 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:37.176010 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:37.176010 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:37.176010 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:37.176010 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:37.176010 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:54:37.176010 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:37.176010 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 390/799] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:54:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:37.179141 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4733c9f-07e3-4cf9-9680-017c6db4ef30 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:37.247123 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af6697-e412-4bf8-afdb-a7021d0db0cb None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:37.247972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:37.286697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:37.288178 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5563d87-f296-4054-a0ac-106345a1f565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:37.288799 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5563d87-f296-4054-a0ac-106345a1f565 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:37.289078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5563d87-f296-4054-a0ac-106345a1f565 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:37.289349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5563d87-f296-4054-a0ac-106345a1f565 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:37.290227 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5563d87-f296-4054-a0ac-106345a1f565 None admin] RBAC: Adding query filter params (name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:37.290533 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5563d87-f296-4054-a0ac-106345a1f565 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:37.292109 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5563d87-f296-4054-a0ac-106345a1f565 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:37.297063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4733c9f-07e3-4cf9-9680-017c6db4ef30 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:37.302172 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 410/800] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 14 01:54:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:37.307978 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5563d87-f296-4054-a0ac-106345a1f565 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:37.334565 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7722669-ef21-4360-9e67-0787e6a580ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:37.335488 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7722669-ef21-4360-9e67-0787e6a580ed None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:37.335951 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7722669-ef21-4360-9e67-0787e6a580ed None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:37.336426 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7722669-ef21-4360-9e67-0787e6a580ed None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:37.337270 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7722669-ef21-4360-9e67-0787e6a580ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:37.337765 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7722669-ef21-4360-9e67-0787e6a580ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:37.339884 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7722669-ef21-4360-9e67-0787e6a580ed None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:37.355583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 391/801] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:54:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:54:37.357830 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7722669-ef21-4360-9e67-0787e6a580ed None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:37.423568 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5563d87-f296-4054-a0ac-106345a1f565 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:37.464295 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33c884-8060-40ee-8f16-e6e5cfe95d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:37.465190 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33c884-8060-40ee-8f16-e6e5cfe95d7e None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:37.465479 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33c884-8060-40ee-8f16-e6e5cfe95d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:37.465751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33c884-8060-40ee-8f16-e6e5cfe95d7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:37.470702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7722669-ef21-4360-9e67-0787e6a580ed None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:37.482464 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33c884-8060-40ee-8f16-e6e5cfe95d7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:37.483949 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33c884-8060-40ee-8f16-e6e5cfe95d7e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:37.501482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-cb33c884-8060-40ee-8f16-e6e5cfe95d7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:37.501482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:37.501482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:37.501482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:37.501482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:37.501482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:37.501482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:37.501482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:37.501482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:37.501482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:37.501482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:37.501482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:37.501482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:37.501482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:37.501482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:37.501482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:37.502819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:37.502819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:37.502819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:37.502819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:37.502819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:37.502819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:37.502819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:37.502819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:37.502819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:37.502819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:37.502819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:37.502819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:37.502819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:37.502819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:37.502819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:37.502819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:37.502819 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:37.504109 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:37.504109 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:37.504109 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:37.504109 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 411/802] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:37.508682 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33c884-8060-40ee-8f16-e6e5cfe95d7e None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:37.535248 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:37.567798 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad23b00-9459-4527-8206-ccfa9d95d848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:37.568371 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad23b00-9459-4527-8206-ccfa9d95d848 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:37.570663 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad23b00-9459-4527-8206-ccfa9d95d848 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:37.570991 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad23b00-9459-4527-8206-ccfa9d95d848 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:37.598675 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad23b00-9459-4527-8206-ccfa9d95d848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:37.600146 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad23b00-9459-4527-8206-ccfa9d95d848 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:37.616610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:37.622997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-1ad23b00-9459-4527-8206-ccfa9d95d848 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:37.622997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:37.622997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:37.622997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:37.622997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:37.622997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:37.622997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:37.622997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:37.622997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:37.622997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:37.622997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:37.622997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:37.622997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:37.622997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:37.622997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:37.622997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:37.623988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:37.623988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:37.623988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:37.623988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:37.623988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:37.623988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:37.623988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:37.623988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:37.623988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:37.623988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:37.623988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:37.623988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:37.623988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:37.623988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:37.623988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:37.623988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:37.623988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:37.624906 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:37.624906 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:37.624906 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:37.631534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 392/803] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:37.637178 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad23b00-9459-4527-8206-ccfa9d95d848 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:37.699987 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33c884-8060-40ee-8f16-e6e5cfe95d7e None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:37.743630 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331b516-03c7-42b1-943c-d888bfe93276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:37.744689 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331b516-03c7-42b1-943c-d888bfe93276 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:37.744971 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331b516-03c7-42b1-943c-d888bfe93276 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:37.745246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331b516-03c7-42b1-943c-d888bfe93276 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:37.746303 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331b516-03c7-42b1-943c-d888bfe93276 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:37.746622 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331b516-03c7-42b1-943c-d888bfe93276 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:37.748582 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331b516-03c7-42b1-943c-d888bfe93276 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:37.764887 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 412/804] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:37.766477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa4dd8-32e2-4aff-ae22-3836debfd21f None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:37.766822 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa4dd8-32e2-4aff-ae22-3836debfd21f None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:37.767097 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa4dd8-32e2-4aff-ae22-3836debfd21f None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:37.772707 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 413/805] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:37.773639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa4dd8-32e2-4aff-ae22-3836debfd21f None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:37.779152 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad23b00-9459-4527-8206-ccfa9d95d848 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:37.822710 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb70ad-3c95-4cfe-99ef-8fdf5166b9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:37.823248 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb70ad-3c95-4cfe-99ef-8fdf5166b9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:37.823530 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb70ad-3c95-4cfe-99ef-8fdf5166b9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:37.823805 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb70ad-3c95-4cfe-99ef-8fdf5166b9a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:37.824786 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb70ad-3c95-4cfe-99ef-8fdf5166b9a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:37.825151 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb70ad-3c95-4cfe-99ef-8fdf5166b9a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:37.826911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb70ad-3c95-4cfe-99ef-8fdf5166b9a9 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:37.848007 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 393/806] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:37.848007 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e9c2c-e27f-456d-87d9-edd1c6db9e35 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:37.848007 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e9c2c-e27f-456d-87d9-edd1c6db9e35 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:37.848007 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e9c2c-e27f-456d-87d9-edd1c6db9e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:37.867369 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-a27e9c2c-e27f-456d-87d9-edd1c6db9e35 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:37.869831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-a27e9c2c-e27f-456d-87d9-edd1c6db9e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:37.898703 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa4dd8-32e2-4aff-ae22-3836debfd21f None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:37.909087 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:37.909854 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:37.942845 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9807fe0-b61d-4e7a-b384-31ea6a4d9e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:37.944385 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9807fe0-b61d-4e7a-b384-31ea6a4d9e79 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:37.944910 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9807fe0-b61d-4e7a-b384-31ea6a4d9e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:37.945631 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9807fe0-b61d-4e7a-b384-31ea6a4d9e79 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:37.956155 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9807fe0-b61d-4e7a-b384-31ea6a4d9e79 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:37.957970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9807fe0-b61d-4e7a-b384-31ea6a4d9e79 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:37.959002 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 394/807] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:54:37.959841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e9c2c-e27f-456d-87d9-edd1c6db9e35 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:37.985279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-c9807fe0-b61d-4e7a-b384-31ea6a4d9e79 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:54:37.985279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:37.985279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:37.985279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:37.985279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:37.985279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:37.985279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:37.985279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:37.985279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:37.985279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:37.985279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:37.985279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:37.985279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:37.985279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:37.985279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:37.985279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:37.986397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:37.986397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:37.986397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:37.986397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:37.986397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:37.986397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:37.986397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:37.986397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:37.986397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:37.986397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:37.986397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:37.986397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:37.986397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:37.986397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:37.986397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:37.986397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:37.986397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:37.986397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:37.988078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:37.988078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:37.988078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:37.988078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:37.988078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:37.988078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:37.988078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:37.988078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:37.988078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:54:37.988078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:37.988078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 414/808] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Sep 14 01:54:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:37.993890 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9807fe0-b61d-4e7a-b384-31ea6a4d9e79 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:38.146173 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e9c2c-e27f-456d-87d9-edd1c6db9e35 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:38.203326 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9807fe0-b61d-4e7a-b384-31ea6a4d9e79 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:38.208107 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266b339-efb7-43cb-9dc0-88cfd2515642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:38.208735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266b339-efb7-43cb-9dc0-88cfd2515642 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:38.209011 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266b339-efb7-43cb-9dc0-88cfd2515642 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:38.209274 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266b339-efb7-43cb-9dc0-88cfd2515642 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:38.222011 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266b339-efb7-43cb-9dc0-88cfd2515642 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:38.223462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266b339-efb7-43cb-9dc0-88cfd2515642 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:38.240192 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-7266b339-efb7-43cb-9dc0-88cfd2515642 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:54:38.240192 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:38.240192 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:38.240192 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:38.240192 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:38.240192 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:38.240192 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:38.240192 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:38.240192 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:38.240192 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:38.240192 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:38.240192 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:38.240192 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:38.240192 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:38.240192 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:38.240192 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:38.241229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:38.241229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:38.241229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:38.241229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:38.241229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:38.241229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:38.241229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:38.241229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:38.241229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:38.241229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:38.241229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:38.241229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:38.241229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:38.241229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:38.241229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:38.241229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:38.241229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:38.241229 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:38.242791 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:38.242791 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:38.242791 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:38.242791 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:38.242791 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:38.242791 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:38.242791 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:38.242791 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:38.242791 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:54:38.242791 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:38.242791 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 395/809] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Sep 14 01:54:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:38.243400 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266b339-efb7-43cb-9dc0-88cfd2515642 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:38.266062 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068a2fe-4252-4a1c-8410-9f8931897e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:38.267372 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068a2fe-4252-4a1c-8410-9f8931897e82 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:38.268066 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068a2fe-4252-4a1c-8410-9f8931897e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:38.268388 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068a2fe-4252-4a1c-8410-9f8931897e82 None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:38.269389 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068a2fe-4252-4a1c-8410-9f8931897e82 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:38.269696 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068a2fe-4252-4a1c-8410-9f8931897e82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:38.271641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068a2fe-4252-4a1c-8410-9f8931897e82 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:38.278364 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:38.293493 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 415/810] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Sep 14 01:54:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:38.294804 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068a2fe-4252-4a1c-8410-9f8931897e82 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:38.326991 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:38.382242 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266b339-efb7-43cb-9dc0-88cfd2515642 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:38.417151 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861399c5-9b2f-4774-8001-2870bd6091a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:38.417711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861399c5-9b2f-4774-8001-2870bd6091a5 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:38.418014 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861399c5-9b2f-4774-8001-2870bd6091a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:38.418310 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861399c5-9b2f-4774-8001-2870bd6091a5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:38.418870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861399c5-9b2f-4774-8001-2870bd6091a5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:38.420233 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861399c5-9b2f-4774-8001-2870bd6091a5 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:38.421132 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068a2fe-4252-4a1c-8410-9f8931897e82 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:38.429649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-861399c5-9b2f-4774-8001-2870bd6091a5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 01:54:38.429649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:38.429649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:54:38.429649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:54:38.429649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:54:38.429649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:54:38.429649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:54:38.429649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:38.429649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 01:54:38.429649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:38.429649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:54:38.429649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:38.429649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:38.429649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:38.429649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:38.429649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:38.429649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:38.430719 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:38.430719 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:38.430719 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:38.430719 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:38.430719 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:38.430719 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:38.430719 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:54:38.430719 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:54:38.430719 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:54:38.430719 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:54:38.430719 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:38.430719 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:38.430719 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:38.430719 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:38.430719 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:38.430719 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:38.431585 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:38.431585 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:38.431585 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:38.431585 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:38.431585 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:38.431585 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:38.431585 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:38.431585 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:38.431585 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:38.431585 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:38.431585 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:54:38.431585 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:38.431585 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 01:54:38.431585 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:38.434263 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 396/811] 10.222.0.15 () {56 vars in 1129 bytes} [Wed Sep 14 01:54:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:38.435088 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861399c5-9b2f-4774-8001-2870bd6091a5 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:38.468776 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae08e46-8093-47e2-9def-91bf177b8ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:38.470605 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae08e46-8093-47e2-9def-91bf177b8ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:38.471038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae08e46-8093-47e2-9def-91bf177b8ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:38.471462 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae08e46-8093-47e2-9def-91bf177b8ab9 None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:38.472590 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae08e46-8093-47e2-9def-91bf177b8ab9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:38.473926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae08e46-8093-47e2-9def-91bf177b8ab9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:38.475562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae08e46-8093-47e2-9def-91bf177b8ab9 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:38.504456 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 416/812] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Sep 14 01:54:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:38.504456 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae08e46-8093-47e2-9def-91bf177b8ab9 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:38.552054 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:38.565304 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861399c5-9b2f-4774-8001-2870bd6091a5 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:38.601385 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44644de4-aebd-448d-b345-611fd4470a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:38.602044 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44644de4-aebd-448d-b345-611fd4470a3c None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:38.602326 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44644de4-aebd-448d-b345-611fd4470a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:38.602594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44644de4-aebd-448d-b345-611fd4470a3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:38.614440 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44644de4-aebd-448d-b345-611fd4470a3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:38.615493 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:38.616275 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44644de4-aebd-448d-b345-611fd4470a3c None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:38.631032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-44644de4-aebd-448d-b345-611fd4470a3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:38.631032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:38.631032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:38.631032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:38.631032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:38.631032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:38.631032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:38.631032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:38.631032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:38.631032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:38.631032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:38.631032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:38.631032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:38.631032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:38.631032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:38.631032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:38.632029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:38.632029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:38.632029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:38.632029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:38.632029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:38.632029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:38.632029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:38.632029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:38.632029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:38.632029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:38.632029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:38.632029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:38.632029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:38.632029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:38.632029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:38.632029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:38.632029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:38.632950 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:38.632950 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:38.632950 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:38.636892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 397/813] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:38.637744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44644de4-aebd-448d-b345-611fd4470a3c None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:38.643189 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae08e46-8093-47e2-9def-91bf177b8ab9 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:38.690539 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed56c30-9f87-4a29-b531-586dd788d6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:38.691069 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed56c30-9f87-4a29-b531-586dd788d6e5 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:38.691343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed56c30-9f87-4a29-b531-586dd788d6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:38.691610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed56c30-9f87-4a29-b531-586dd788d6e5 None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:38.692228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed56c30-9f87-4a29-b531-586dd788d6e5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:38.692528 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed56c30-9f87-4a29-b531-586dd788d6e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:38.693893 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed56c30-9f87-4a29-b531-586dd788d6e5 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:38.703813 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 417/814] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Sep 14 01:54:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:38.705274 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed56c30-9f87-4a29-b531-586dd788d6e5 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:38.762055 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44644de4-aebd-448d-b345-611fd4470a3c None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:38.811378 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37257225-9fdd-4d07-90ac-db0f1d57f8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:38.811987 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37257225-9fdd-4d07-90ac-db0f1d57f8b3 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:38.812342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37257225-9fdd-4d07-90ac-db0f1d57f8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:38.812575 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37257225-9fdd-4d07-90ac-db0f1d57f8b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:38.823472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed56c30-9f87-4a29-b531-586dd788d6e5 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:38.823957 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:38.866348 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4d5fd-7bfd-4282-9349-f9ca484b28a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:38.867364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4d5fd-7bfd-4282-9349-f9ca484b28a6 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:38.867969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4d5fd-7bfd-4282-9349-f9ca484b28a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:38.868443 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4d5fd-7bfd-4282-9349-f9ca484b28a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:38.869279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae4d5fd-7bfd-4282-9349-f9ca484b28a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:38.869861 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae4d5fd-7bfd-4282-9349-f9ca484b28a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:38.871658 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae4d5fd-7bfd-4282-9349-f9ca484b28a6 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:38.884875 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37257225-9fdd-4d07-90ac-db0f1d57f8b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:38.885425 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 418/815] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:38.886475 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4d5fd-7bfd-4282-9349-f9ca484b28a6 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:38.890423 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37257225-9fdd-4d07-90ac-db0f1d57f8b3 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:38.903790 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:39.014053 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4d5fd-7bfd-4282-9349-f9ca484b28a6 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:39.039034 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 398/816] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:54:38 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:54:39.039034 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37257225-9fdd-4d07-90ac-db0f1d57f8b3 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:39.060418 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43741d-b4cf-4090-9128-dc40a7ca27d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:39.061862 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43741d-b4cf-4090-9128-dc40a7ca27d3 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:39.062386 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43741d-b4cf-4090-9128-dc40a7ca27d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:39.062894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43741d-b4cf-4090-9128-dc40a7ca27d3 None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:39.063827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43741d-b4cf-4090-9128-dc40a7ca27d3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:39.064885 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43741d-b4cf-4090-9128-dc40a7ca27d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:39.067057 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43741d-b4cf-4090-9128-dc40a7ca27d3 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:39.079130 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 419/817] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Sep 14 01:54:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:39.079130 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43741d-b4cf-4090-9128-dc40a7ca27d3 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:39.092074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:39.153762 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37257225-9fdd-4d07-90ac-db0f1d57f8b3 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:39.180820 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:39.192362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9936b-4b55-4c75-9f5e-a5cd3844f536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:39.193006 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9936b-4b55-4c75-9f5e-a5cd3844f536 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:39.193402 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9936b-4b55-4c75-9f5e-a5cd3844f536 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:39.193803 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9936b-4b55-4c75-9f5e-a5cd3844f536 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:39.201027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9936b-4b55-4c75-9f5e-a5cd3844f536 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:39.202604 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9936b-4b55-4c75-9f5e-a5cd3844f536 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:39.214757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-02c9936b-4b55-4c75-9f5e-a5cd3844f536 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:39.214757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:39.214757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:39.214757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:39.214757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:39.214757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:39.214757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:39.214757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:39.214757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:39.214757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:39.214757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:39.214757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:39.214757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:39.214757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:39.214757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:39.215894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:39.215894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:39.215894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:39.215894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:39.215894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:39.215894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:39.215894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:39.215894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:39.215894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:39.215894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:39.215894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:39.215894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:39.215894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:39.215894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:39.215894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:39.215894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:39.215894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:39.217040 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:39.217040 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:39.217040 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:39.217040 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:39.217040 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:39.217040 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:39.217040 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:39.217040 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:39.222096 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 399/818] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:39.223020 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9936b-4b55-4c75-9f5e-a5cd3844f536 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:39.224793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43741d-b4cf-4090-9128-dc40a7ca27d3 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:39.275906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee121748-4b42-421c-81a4-edf2ece3cf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:39.277086 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee121748-4b42-421c-81a4-edf2ece3cf69 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:39.277518 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee121748-4b42-421c-81a4-edf2ece3cf69 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:39.277962 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee121748-4b42-421c-81a4-edf2ece3cf69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:39.291887 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee121748-4b42-421c-81a4-edf2ece3cf69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:39.293489 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee121748-4b42-421c-81a4-edf2ece3cf69 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:39.308175 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-ee121748-4b42-421c-81a4-edf2ece3cf69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:39.308175 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:39.308175 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:39.308175 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:39.308175 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:39.308175 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:39.308175 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:39.308175 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:39.308175 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:39.308175 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:39.308175 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:39.308175 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:39.308175 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:39.308175 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:39.308175 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:39.308175 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:39.309367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:39.309367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:39.309367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:39.309367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:39.309367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:39.309367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:39.309367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:39.309367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:39.309367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:39.309367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:39.309367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:39.309367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:39.309367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:39.309367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:39.309367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:39.309367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:39.309367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:39.310370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:39.310370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:39.310370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:39.315700 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 420/819] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:39.316617 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee121748-4b42-421c-81a4-edf2ece3cf69 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:39.351207 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:39.358603 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9936b-4b55-4c75-9f5e-a5cd3844f536 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:39.396146 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94670ab-b91d-48b0-8a74-22c150e173b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:39.396759 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94670ab-b91d-48b0-8a74-22c150e173b5 None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:39.397036 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94670ab-b91d-48b0-8a74-22c150e173b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:39.397306 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94670ab-b91d-48b0-8a74-22c150e173b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:39.397966 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94670ab-b91d-48b0-8a74-22c150e173b5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:39.399357 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94670ab-b91d-48b0-8a74-22c150e173b5 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:39.423624 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 400/820] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 14 01:54:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:54:39.424714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94670ab-b91d-48b0-8a74-22c150e173b5 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:39.451449 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee121748-4b42-421c-81a4-edf2ece3cf69 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:39.493260 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:39.517968 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70fc79-ea95-4e3a-a6de-0a135a87c3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:39.519018 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70fc79-ea95-4e3a-a6de-0a135a87c3b8 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:39.519569 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70fc79-ea95-4e3a-a6de-0a135a87c3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:39.520081 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70fc79-ea95-4e3a-a6de-0a135a87c3b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:39.521098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d70fc79-ea95-4e3a-a6de-0a135a87c3b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:39.521659 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d70fc79-ea95-4e3a-a6de-0a135a87c3b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:39.523425 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d70fc79-ea95-4e3a-a6de-0a135a87c3b8 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:39.539247 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 421/821] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:39.540898 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2c832-f020-44e0-a82a-8d54ecc1d0d7 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:39.541392 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2c832-f020-44e0-a82a-8d54ecc1d0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:39.541898 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2c832-f020-44e0-a82a-8d54ecc1d0d7 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:39.543615 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 422/822] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:39.544521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2c832-f020-44e0-a82a-8d54ecc1d0d7 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:39.581708 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94670ab-b91d-48b0-8a74-22c150e173b5 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:39.624906 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134cc2c-650f-471f-b262-ca1b7f1a2725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:39.625523 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134cc2c-650f-471f-b262-ca1b7f1a2725 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:39.625806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134cc2c-650f-471f-b262-ca1b7f1a2725 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:39.626677 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134cc2c-650f-471f-b262-ca1b7f1a2725 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:39.627331 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6134cc2c-650f-471f-b262-ca1b7f1a2725 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:39.627974 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6134cc2c-650f-471f-b262-ca1b7f1a2725 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:39.629394 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6134cc2c-650f-471f-b262-ca1b7f1a2725 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:39.642314 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 401/823] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:39.643653 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4764c04-b121-4698-9f82-76b37ca08e8c None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:39.644027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4764c04-b121-4698-9f82-76b37ca08e8c None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:39.644367 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4764c04-b121-4698-9f82-76b37ca08e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:39.662434 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-d4764c04-b121-4698-9f82-76b37ca08e8c None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:39.663517 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-d4764c04-b121-4698-9f82-76b37ca08e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:39.674648 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2c832-f020-44e0-a82a-8d54ecc1d0d7 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:39.677788 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:39.710170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e63a3-2c78-466f-9980-165f9f817a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:39.710838 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e63a3-2c78-466f-9980-165f9f817a35 None admin] REQUEST_METHOD: `PUT` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:39.711118 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e63a3-2c78-466f-9980-165f9f817a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:39.711393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e63a3-2c78-466f-9980-165f9f817a35 None admin] PATH_INFO: `/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/e1c8d2eec886482eb135e19ded2c61ae/roles/e962a2a280224f9fbcd0fdc02262993d` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:39.726502 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e63a3-2c78-466f-9980-165f9f817a35 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e04b16dba1c4de79dc6b1a5e16ed907, user_id=e1c8d2eec886482eb135e19ded2c61ae, role_id=e962a2a280224f9fbcd0fdc02262993d)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:39.727947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e63a3-2c78-466f-9980-165f9f817a35 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:39.740613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 402/824] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:54:39.741747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 423/825] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:54:39 2022] PUT /identity/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/e1c8d2eec886482eb135e19ded2c61ae/roles/e962a2a280224f9fbcd0fdc02262993d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:54:39.742301 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90fe503-7650-43ae-846a-1802a72661f7 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:39.742595 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e63a3-2c78-466f-9980-165f9f817a35 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:39.742996 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90fe503-7650-43ae-846a-1802a72661f7 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:39.743447 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90fe503-7650-43ae-846a-1802a72661f7 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:39.745144 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 403/826] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:39.750750 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf71bde-43e2-48f1-bc79-9257a7e67a26 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:39.751894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf71bde-43e2-48f1-bc79-9257a7e67a26 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:39.752344 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf71bde-43e2-48f1-bc79-9257a7e67a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:39.789641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-8bf71bde-43e2-48f1-bc79-9257a7e67a26 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:39.792284 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-8bf71bde-43e2-48f1-bc79-9257a7e67a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:39.807143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:39.924501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 404/827] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:54:39.931067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e63a3-2c78-466f-9980-165f9f817a35 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:39.973959 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44671e-08b2-4b54-a14e-1ee7b6404aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:39.974633 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44671e-08b2-4b54-a14e-1ee7b6404aaa None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:39.974916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44671e-08b2-4b54-a14e-1ee7b6404aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:39.975185 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44671e-08b2-4b54-a14e-1ee7b6404aaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:39.985233 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44671e-08b2-4b54-a14e-1ee7b6404aaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:39.986786 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44671e-08b2-4b54-a14e-1ee7b6404aaa None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:40.002003 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-ed44671e-08b2-4b54-a14e-1ee7b6404aaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:40.002003 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:40.002003 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:40.002003 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:40.002003 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:40.002003 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:40.002003 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:40.002003 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:40.002003 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:40.002003 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:40.002003 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:40.002003 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:40.002003 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:40.002003 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:40.002003 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:40.002940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:40.002940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:40.002940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:40.002940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:40.002940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:40.002940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:40.002940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:40.002940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:40.002940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:40.002940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:40.002940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:40.002940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:40.002940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:40.002940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:40.002940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:40.002940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:40.002940 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:40.004451 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:40.004451 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:40.004451 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:40.004451 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:40.004451 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:40.004451 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:40.004451 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:40.004451 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:40.004451 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 424/828] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:40.009497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf71bde-43e2-48f1-bc79-9257a7e67a26 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:40.059079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44671e-08b2-4b54-a14e-1ee7b6404aaa None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:40.104166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:40.147132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf71bde-43e2-48f1-bc79-9257a7e67a26 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:40.194684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:40.208328 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44671e-08b2-4b54-a14e-1ee7b6404aaa None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:40.235608 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436873d-210a-4b97-a115-79a7b986d2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:40.236288 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436873d-210a-4b97-a115-79a7b986d2ae None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:40.236567 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436873d-210a-4b97-a115-79a7b986d2ae None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:40.236838 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436873d-210a-4b97-a115-79a7b986d2ae None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:40.237522 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436873d-210a-4b97-a115-79a7b986d2ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:40.237824 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436873d-210a-4b97-a115-79a7b986d2ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:40.239483 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436873d-210a-4b97-a115-79a7b986d2ae None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:40.244272 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a85671-a33c-47be-b7a3-c7b499d87110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:40.245016 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a85671-a33c-47be-b7a3-c7b499d87110 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:40.245395 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a85671-a33c-47be-b7a3-c7b499d87110 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:40.245798 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a85671-a33c-47be-b7a3-c7b499d87110 None admin] PATH_INFO: `/v3/services/network` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:40.246547 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a85671-a33c-47be-b7a3-c7b499d87110 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:40.248003 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a85671-a33c-47be-b7a3-c7b499d87110 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:40.252766 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 405/829] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:40.253933 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436873d-210a-4b97-a115-79a7b986d2ae None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:40.260781 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-60a85671-a33c-47be-b7a3-c7b499d87110 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 01:54:40.260781 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:40.260781 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:54:40.260781 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:54:40.260781 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:54:40.260781 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:54:40.260781 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:54:40.260781 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:40.260781 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 01:54:40.260781 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:40.260781 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:54:40.260781 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:40.260781 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:40.260781 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:40.260781 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:40.260781 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:40.260781 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:40.261821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:40.261821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:40.261821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:40.261821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:40.261821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:40.261821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:40.261821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:54:40.261821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:54:40.261821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:54:40.261821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:54:40.261821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:40.261821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:40.261821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:40.261821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:40.261821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:40.261821 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:40.262751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:40.262751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:40.262751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:40.262751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:40.262751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:40.262751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:40.262751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:40.262751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:40.262751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:40.262751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:40.262751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:54:40.262751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:40.262751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 01:54:40.262751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:40.262751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 425/830] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:40 2022] GET /identity/v3/services/network => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:40.263691 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a85671-a33c-47be-b7a3-c7b499d87110 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:40.365656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a85671-a33c-47be-b7a3-c7b499d87110 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:40.366950 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436873d-210a-4b97-a115-79a7b986d2ae None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:40.402551 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdacd4-3528-4d88-921d-252e6dc547c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:40.403164 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdacd4-3528-4d88-921d-252e6dc547c3 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:40.403439 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdacd4-3528-4d88-921d-252e6dc547c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:40.403845 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdacd4-3528-4d88-921d-252e6dc547c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:40.404584 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebdacd4-3528-4d88-921d-252e6dc547c3 None admin] RBAC: Adding query filter params (role.id=aa420c2fb1ab4bb3b05b004a4d89779a, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=f721a3a4eca941989685448a87bb9a19) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:40.404891 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebdacd4-3528-4d88-921d-252e6dc547c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:40.406975 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebdacd4-3528-4d88-921d-252e6dc547c3 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:40.410536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068003b-335e-4300-a840-174463e99266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:40.411418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068003b-335e-4300-a840-174463e99266 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:40.411910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068003b-335e-4300-a840-174463e99266 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:40.412380 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068003b-335e-4300-a840-174463e99266 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:40.418007 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 426/831] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:54:40 2022] GET /identity/v3/role_assignments?user.id=f721a3a4eca941989685448a87bb9a19&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=aa420c2fb1ab4bb3b05b004a4d89779a => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:40.418007 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdacd4-3528-4d88-921d-252e6dc547c3 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:40.423684 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:40.436232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:40.450365 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6068003b-335e-4300-a840-174463e99266 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:40.457008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-6068003b-335e-4300-a840-174463e99266 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:54:40.457008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:40.457008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:40.457008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:40.457008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:40.457008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:40.457008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:40.457008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:40.457008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:40.457008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:40.457008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:40.457008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:40.457008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:40.457008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:40.457008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:40.457008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:40.457963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:40.457963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:40.457963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:40.457963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:40.457963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:40.457963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:40.457963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:40.457963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:40.457963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:40.457963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:40.457963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:40.457963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:40.457963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:40.457963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:40.457963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:40.457963 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:40.458902 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:40.458902 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:40.458902 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:40.458902 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:40.458902 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:40.458902 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:40.458902 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:40.458902 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:40.458902 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:40.458902 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:40.458902 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:54:40.458902 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:40.466762 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 406/832] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:54:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:40.473272 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068003b-335e-4300-a840-174463e99266 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:40.535486 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdacd4-3528-4d88-921d-252e6dc547c3 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:40.576997 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8096895-b39d-4be4-b335-1e19579650e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:40.577647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8096895-b39d-4be4-b335-1e19579650e3 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:40.577923 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8096895-b39d-4be4-b335-1e19579650e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:40.578226 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8096895-b39d-4be4-b335-1e19579650e3 None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:40.579837 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8096895-b39d-4be4-b335-1e19579650e3 None admin] RBAC: Adding query filter params (name=network) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:40.580150 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8096895-b39d-4be4-b335-1e19579650e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:40.581684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8096895-b39d-4be4-b335-1e19579650e3 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:40.587222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 427/833] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 14 01:54:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:40.595357 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8096895-b39d-4be4-b335-1e19579650e3 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:40.679522 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068003b-335e-4300-a840-174463e99266 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:40.754558 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:40.764755 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb7f94-65bb-4d97-9c9b-0a1639d01ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:40.766081 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb7f94-65bb-4d97-9c9b-0a1639d01ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:40.766907 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb7f94-65bb-4d97-9c9b-0a1639d01ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:40.767394 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb7f94-65bb-4d97-9c9b-0a1639d01ae2 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:40.768253 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb7f94-65bb-4d97-9c9b-0a1639d01ae2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:40.768359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8096895-b39d-4be4-b335-1e19579650e3 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:40.768847 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb7f94-65bb-4d97-9c9b-0a1639d01ae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:40.771396 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb7f94-65bb-4d97-9c9b-0a1639d01ae2 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:40.783041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 407/834] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:54:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:40.788941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb7f94-65bb-4d97-9c9b-0a1639d01ae2 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:40.813863 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb4f89-58dd-4af9-b456-5037b4c38bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:40.814561 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb4f89-58dd-4af9-b456-5037b4c38bfd None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:40.814844 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb4f89-58dd-4af9-b456-5037b4c38bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:40.815118 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb4f89-58dd-4af9-b456-5037b4c38bfd None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:40.815752 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb4f89-58dd-4af9-b456-5037b4c38bfd None admin] RBAC: Adding query filter params (type=network) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:40.816050 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb4f89-58dd-4af9-b456-5037b4c38bfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:40.817919 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb4f89-58dd-4af9-b456-5037b4c38bfd None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:40.824626 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 428/835] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 14 01:54:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:40.831128 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb4f89-58dd-4af9-b456-5037b4c38bfd None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:40.893429 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:40.969499 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb7f94-65bb-4d97-9c9b-0a1639d01ae2 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:40.998001 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb4f89-58dd-4af9-b456-5037b4c38bfd None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:41.014906 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5d920-97c2-4a6d-8403-311484ed8429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:41.015551 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5d920-97c2-4a6d-8403-311484ed8429 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:41.015829 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5d920-97c2-4a6d-8403-311484ed8429 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:41.016101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5d920-97c2-4a6d-8403-311484ed8429 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:41.036799 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5d920-97c2-4a6d-8403-311484ed8429 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:41.038605 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5d920-97c2-4a6d-8403-311484ed8429 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:41.042184 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54e2be-b95f-477e-90f4-81d59a37b359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:41.043030 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54e2be-b95f-477e-90f4-81d59a37b359 None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:41.043503 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54e2be-b95f-477e-90f4-81d59a37b359 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:41.043963 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54e2be-b95f-477e-90f4-81d59a37b359 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:41.044870 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54e2be-b95f-477e-90f4-81d59a37b359 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:41.046450 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54e2be-b95f-477e-90f4-81d59a37b359 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:41.067599 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-d4d5d920-97c2-4a6d-8403-311484ed8429 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:41.067599 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:41.067599 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:41.067599 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:41.067599 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:41.067599 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:41.067599 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:41.067599 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:41.067599 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:41.067599 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:41.067599 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:41.067599 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:41.067599 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:41.067599 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:41.067599 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:41.067599 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:41.068594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:41.068594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:41.068594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:41.068594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:41.068594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:41.068594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:41.068594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:41.068594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:41.068594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:41.068594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:41.068594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:41.068594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:41.068594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:41.068594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:41.068594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:41.068594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:41.068594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:41.069508 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:41.069508 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:41.069508 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:41.076210 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 408/836] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:41.081683 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:41.084719 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5d920-97c2-4a6d-8403-311484ed8429 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:41.091651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 429/837] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 14 01:54:40 2022] POST /identity/v3/endpoints => generated 337 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:54:41.222794 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5d920-97c2-4a6d-8403-311484ed8429 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:41.236971 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:41.269421 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4e774-9505-4c0e-88cc-bb2aeffd530d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:41.270220 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4e774-9505-4c0e-88cc-bb2aeffd530d None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:41.270494 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4e774-9505-4c0e-88cc-bb2aeffd530d None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:41.270823 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4e774-9505-4c0e-88cc-bb2aeffd530d None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:41.271516 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4e774-9505-4c0e-88cc-bb2aeffd530d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:41.271814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4e774-9505-4c0e-88cc-bb2aeffd530d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:41.273182 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4e774-9505-4c0e-88cc-bb2aeffd530d None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:41.286188 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 409/838] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:41.295152 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54e2be-b95f-477e-90f4-81d59a37b359 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:41.424117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54e2be-b95f-477e-90f4-81d59a37b359 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:41.461529 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfc1f2-6b83-4445-8e88-2814541eeed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:41.462613 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfc1f2-6b83-4445-8e88-2814541eeed4 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:41.463093 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfc1f2-6b83-4445-8e88-2814541eeed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:41.463577 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfc1f2-6b83-4445-8e88-2814541eeed4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:41.472249 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdfc1f2-6b83-4445-8e88-2814541eeed4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:41.474030 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdfc1f2-6b83-4445-8e88-2814541eeed4 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:41.485685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-9fdfc1f2-6b83-4445-8e88-2814541eeed4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:54:41.485685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:41.485685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:41.485685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:41.485685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:41.485685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:41.485685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:41.485685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:41.485685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:41.485685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:41.485685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:41.485685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:41.485685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:41.485685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:41.485685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:41.485685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:41.486674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:41.486674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:41.486674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:41.486674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:41.486674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:41.486674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:41.486674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:41.486674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:41.486674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:41.486674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:41.486674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:41.486674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:41.486674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:41.486674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:41.486674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:41.486674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:41.486674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:41.486674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:41.487652 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:41.487652 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:41.487652 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:41.487652 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:41.487652 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:41.487652 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:41.487652 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:41.487652 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:41.487652 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:54:41.487652 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:41.490108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 430/839] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:54:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:41.498106 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4e774-9505-4c0e-88cc-bb2aeffd530d None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:41.617349 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4e774-9505-4c0e-88cc-bb2aeffd530d None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:41.656699 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b420c9-b6a5-4c98-89cb-54d519bfab4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:41.657627 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b420c9-b6a5-4c98-89cb-54d519bfab4d None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:41.658077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b420c9-b6a5-4c98-89cb-54d519bfab4d None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:41.658490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b420c9-b6a5-4c98-89cb-54d519bfab4d None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:41.659381 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b420c9-b6a5-4c98-89cb-54d519bfab4d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:41.659944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b420c9-b6a5-4c98-89cb-54d519bfab4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:41.661584 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b420c9-b6a5-4c98-89cb-54d519bfab4d None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:41.685059 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 410/840] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Sep 14 01:54:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:41.692065 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfc1f2-6b83-4445-8e88-2814541eeed4 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:41.766930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:41.806723 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfc1f2-6b83-4445-8e88-2814541eeed4 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:41.845789 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ba48f-45d5-4039-b36f-7dc75bf1dc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:41.846424 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ba48f-45d5-4039-b36f-7dc75bf1dc2f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:41.846699 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ba48f-45d5-4039-b36f-7dc75bf1dc2f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:41.846967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ba48f-45d5-4039-b36f-7dc75bf1dc2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:41.932875 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ba48f-45d5-4039-b36f-7dc75bf1dc2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:41.934723 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ba48f-45d5-4039-b36f-7dc75bf1dc2f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:41.995777 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:42.009970 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f71f0-fcd7-4d1e-beb9-2040b3d4111e None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:42.010282 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f71f0-fcd7-4d1e-beb9-2040b3d4111e None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:42.010790 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f71f0-fcd7-4d1e-beb9-2040b3d4111e None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:42.012579 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 411/841] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:42.018482 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a943f4-bf4f-431e-b386-6caa4573a060 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:42.018700 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a943f4-bf4f-431e-b386-6caa4573a060 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:42.018975 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a943f4-bf4f-431e-b386-6caa4573a060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:42.036701 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-42a943f4-bf4f-431e-b386-6caa4573a060 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:42.037886 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-42a943f4-bf4f-431e-b386-6caa4573a060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:42.046716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 431/842] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:54:41 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:54:42.052346 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ba48f-45d5-4039-b36f-7dc75bf1dc2f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:42.066537 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:42.134018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 412/843] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:54:42.171089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ba48f-45d5-4039-b36f-7dc75bf1dc2f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:42.211329 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963868d7-e831-41ce-aea1-f3b6fbf7bc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:42.211963 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963868d7-e831-41ce-aea1-f3b6fbf7bc49 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:42.212239 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963868d7-e831-41ce-aea1-f3b6fbf7bc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:42.212518 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963868d7-e831-41ce-aea1-f3b6fbf7bc49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:42.233429 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963868d7-e831-41ce-aea1-f3b6fbf7bc49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:42.237139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963868d7-e831-41ce-aea1-f3b6fbf7bc49 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:42.252695 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-963868d7-e831-41ce-aea1-f3b6fbf7bc49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:42.252695 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:42.252695 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:42.252695 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:42.252695 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:42.252695 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:42.252695 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:42.252695 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:42.252695 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:42.252695 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:42.252695 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:42.252695 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:42.252695 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:42.252695 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:42.252695 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:42.252695 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:42.253692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:42.253692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:42.253692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:42.253692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:42.253692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:42.253692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:42.253692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:42.253692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:42.253692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:42.253692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:42.253692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:42.253692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:42.253692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:42.253692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:42.253692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:42.253692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:42.253692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:42.255569 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:42.255569 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:42.255569 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:42.258115 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 432/844] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:42.259952 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a943f4-bf4f-431e-b386-6caa4573a060 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:42.266701 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1eb810-fc09-4d6a-a275-d7222da02e6b None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:42.267001 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1eb810-fc09-4d6a-a275-d7222da02e6b None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:42.267281 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1eb810-fc09-4d6a-a275-d7222da02e6b None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:42.269307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 433/845] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:42.275378 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5372bc-1366-4452-8bdf-627f761f7d28 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:42.275669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5372bc-1366-4452-8bdf-627f761f7d28 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:42.276350 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5372bc-1366-4452-8bdf-627f761f7d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:42.300234 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-cf5372bc-1366-4452-8bdf-627f761f7d28 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:42.301448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-cf5372bc-1366-4452-8bdf-627f761f7d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:42.311604 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:42.385639 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a943f4-bf4f-431e-b386-6caa4573a060 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:42.394333 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 434/846] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:54:42.416716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5372bc-1366-4452-8bdf-627f761f7d28 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:42.430118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4837e1-062e-4b09-a407-37fa63e11577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:42.430741 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4837e1-062e-4b09-a407-37fa63e11577 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:42.431020 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4837e1-062e-4b09-a407-37fa63e11577 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:42.431289 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4837e1-062e-4b09-a407-37fa63e11577 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:42.432055 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4837e1-062e-4b09-a407-37fa63e11577 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:42.432356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4837e1-062e-4b09-a407-37fa63e11577 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:42.433846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4837e1-062e-4b09-a407-37fa63e11577 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:42.447132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 413/847] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:42.455564 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4837e1-062e-4b09-a407-37fa63e11577 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:42.475197 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:42.533933 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5372bc-1366-4452-8bdf-627f761f7d28 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:42.588034 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4837e1-062e-4b09-a407-37fa63e11577 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:42.626538 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea92f11-e94e-4573-b2a7-7dd8f5717182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:42.627366 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea92f11-e94e-4573-b2a7-7dd8f5717182 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:42.627750 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea92f11-e94e-4573-b2a7-7dd8f5717182 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:42.628127 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea92f11-e94e-4573-b2a7-7dd8f5717182 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:42.636513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea92f11-e94e-4573-b2a7-7dd8f5717182 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:42.638182 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea92f11-e94e-4573-b2a7-7dd8f5717182 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:42.652462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-fea92f11-e94e-4573-b2a7-7dd8f5717182 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:42.652462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:42.652462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:42.652462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:42.652462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:42.652462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:42.652462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:42.652462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:42.652462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:42.652462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:42.652462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:42.652462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:42.652462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:42.652462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:42.652462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:42.653569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:42.653569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:42.653569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:42.653569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:42.653569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:42.653569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:42.653569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:42.653569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:42.653569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:42.653569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:42.653569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:42.653569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:42.653569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:42.653569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:42.653569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:42.653569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:42.653569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:42.654533 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:42.654533 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:42.654533 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:42.654533 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:42.654533 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:42.654533 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:42.654533 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:42.654533 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:42.655309 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 414/848] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:42.663046 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea92f11-e94e-4573-b2a7-7dd8f5717182 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:42.697258 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814827c8-8219-4945-86bd-1ce60b5e89bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:42.697886 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814827c8-8219-4945-86bd-1ce60b5e89bc None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:42.698195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814827c8-8219-4945-86bd-1ce60b5e89bc None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:42.698462 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814827c8-8219-4945-86bd-1ce60b5e89bc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:42.699030 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814827c8-8219-4945-86bd-1ce60b5e89bc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:42.700384 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814827c8-8219-4945-86bd-1ce60b5e89bc None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:42.709643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-814827c8-8219-4945-86bd-1ce60b5e89bc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 01:54:42.709643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:42.709643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:54:42.709643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:54:42.709643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:54:42.709643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:54:42.709643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:54:42.709643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:42.709643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 01:54:42.709643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:42.709643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:54:42.709643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:42.709643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:42.709643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:42.709643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:42.709643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:42.709643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:42.710827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:42.710827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:42.710827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:42.710827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:42.710827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:42.710827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:42.710827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:54:42.710827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:54:42.710827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:54:42.710827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:54:42.710827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:42.710827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:42.710827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:42.710827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:42.710827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:42.710827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:42.711726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:42.711726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:42.711726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:42.711726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:42.711726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:42.711726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:42.711726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:42.711726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:42.711726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:42.711726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:42.711726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:54:42.711726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:42.711726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 01:54:42.711726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:42.711726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 435/849] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:54:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:42.712574 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814827c8-8219-4945-86bd-1ce60b5e89bc None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:42.722581 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:42.749484 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:42.777557 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea92f11-e94e-4573-b2a7-7dd8f5717182 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:42.816561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e01051-b00c-44bc-836b-6c7cc6f2fc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:42.817142 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e01051-b00c-44bc-836b-6c7cc6f2fc93 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:42.817367 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e01051-b00c-44bc-836b-6c7cc6f2fc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:42.817633 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e01051-b00c-44bc-836b-6c7cc6f2fc93 None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:42.818345 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e01051-b00c-44bc-836b-6c7cc6f2fc93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:42.818644 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e01051-b00c-44bc-836b-6c7cc6f2fc93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:42.820005 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814827c8-8219-4945-86bd-1ce60b5e89bc None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:42.820118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e01051-b00c-44bc-836b-6c7cc6f2fc93 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:42.833593 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 415/850] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:42.834440 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e01051-b00c-44bc-836b-6c7cc6f2fc93 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:42.861306 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48413ded-1d73-4434-a681-10d9097113f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:42.862259 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48413ded-1d73-4434-a681-10d9097113f5 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:42.862721 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48413ded-1d73-4434-a681-10d9097113f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:42.863073 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48413ded-1d73-4434-a681-10d9097113f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:42.873203 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48413ded-1d73-4434-a681-10d9097113f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:42.877925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-48413ded-1d73-4434-a681-10d9097113f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:42.877925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:42.877925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:42.877925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:42.877925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:42.877925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:42.877925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:42.877925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:42.877925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:42.877925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:42.877925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:42.877925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:42.877925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:42.877925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:42.877925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:42.877925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:42.879015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:42.879015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:42.879015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:42.879015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:42.879015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:42.879015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:42.879015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:42.879015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:42.879015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:42.879015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:42.879015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:42.879015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:42.879015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:42.879015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:42.879015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:42.879015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:42.880812 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:42.880812 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:42.880812 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:42.880812 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:42.880812 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:42.880812 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:42.880812 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:42.880812 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:42.880812 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:42.880812 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:42.880812 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:54:42.880812 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:42.880812 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 436/851] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:54:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:42.880812 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48413ded-1d73-4434-a681-10d9097113f5 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:42.955606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e01051-b00c-44bc-836b-6c7cc6f2fc93 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:42.956605 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:42.991733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48413ded-1d73-4434-a681-10d9097113f5 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:42.994487 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c46fd-6fe1-497f-9f4e-e17a720aa0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:42.995163 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c46fd-6fe1-497f-9f4e-e17a720aa0af None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:42.995441 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c46fd-6fe1-497f-9f4e-e17a720aa0af None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:42.995703 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c46fd-6fe1-497f-9f4e-e17a720aa0af None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:42.996326 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c46fd-6fe1-497f-9f4e-e17a720aa0af None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:42.996620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c46fd-6fe1-497f-9f4e-e17a720aa0af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:42.998122 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c46fd-6fe1-497f-9f4e-e17a720aa0af None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:42.999326 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:43.004393 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 416/852] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:54:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:43.005378 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c46fd-6fe1-497f-9f4e-e17a720aa0af None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:43.032652 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47044543-bfa9-49da-925f-0f08bee27ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:43.033342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47044543-bfa9-49da-925f-0f08bee27ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:43.033629 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47044543-bfa9-49da-925f-0f08bee27ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:43.033901 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47044543-bfa9-49da-925f-0f08bee27ce7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:43.034647 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47044543-bfa9-49da-925f-0f08bee27ce7 None admin] RBAC: Adding query filter params (role.id=aa420c2fb1ab4bb3b05b004a4d89779a, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=21f481c1f60043e9b0e25ba49b30715f) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:43.034957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47044543-bfa9-49da-925f-0f08bee27ce7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:43.036421 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47044543-bfa9-49da-925f-0f08bee27ce7 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:43.042032 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 437/853] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:54:42 2022] GET /identity/v3/role_assignments?user.id=21f481c1f60043e9b0e25ba49b30715f&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=aa420c2fb1ab4bb3b05b004a4d89779a => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:43.043157 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e295df-023d-4d8e-9efb-1083e817655b None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:43.043441 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e295df-023d-4d8e-9efb-1083e817655b None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:43.043714 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e295df-023d-4d8e-9efb-1083e817655b None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:43.044888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 438/854] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:43.045583 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e295df-023d-4d8e-9efb-1083e817655b None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:43.121676 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c46fd-6fe1-497f-9f4e-e17a720aa0af None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:43.159329 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07749ec-ce6b-4529-8fff-446cd1e14bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:43.160080 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07749ec-ce6b-4529-8fff-446cd1e14bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:43.160423 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07749ec-ce6b-4529-8fff-446cd1e14bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:43.160701 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07749ec-ce6b-4529-8fff-446cd1e14bd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:43.161375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07749ec-ce6b-4529-8fff-446cd1e14bd8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:43.161675 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07749ec-ce6b-4529-8fff-446cd1e14bd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:43.163095 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07749ec-ce6b-4529-8fff-446cd1e14bd8 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:43.177027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 417/855] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 14 01:54:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:43.177027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b15ef-2b43-4f51-8e11-7c2a8f5c4413 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:43.177027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b15ef-2b43-4f51-8e11-7c2a8f5c4413 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:43.177027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b15ef-2b43-4f51-8e11-7c2a8f5c4413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:43.187599 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e295df-023d-4d8e-9efb-1083e817655b None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:43.195421 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-c75b15ef-2b43-4f51-8e11-7c2a8f5c4413 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:43.197211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-c75b15ef-2b43-4f51-8e11-7c2a8f5c4413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:43.221664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:43.229590 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20313abc-b823-4e45-862b-e97600abb065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:43.230241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20313abc-b823-4e45-862b-e97600abb065 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:43.230526 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20313abc-b823-4e45-862b-e97600abb065 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:43.230797 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20313abc-b823-4e45-862b-e97600abb065 None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:43.231517 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20313abc-b823-4e45-862b-e97600abb065 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:43.231824 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20313abc-b823-4e45-862b-e97600abb065 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:43.233191 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20313abc-b823-4e45-862b-e97600abb065 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:43.238181 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 439/856] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:54:43 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:43.238980 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20313abc-b823-4e45-862b-e97600abb065 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:43.268458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:43.290390 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 418/857] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:54:43.358537 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20313abc-b823-4e45-862b-e97600abb065 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:43.397902 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f294b6-ff7a-4650-ad23-cb425760d06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:43.398596 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f294b6-ff7a-4650-ad23-cb425760d06a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:43.398874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f294b6-ff7a-4650-ad23-cb425760d06a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:43.399148 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f294b6-ff7a-4650-ad23-cb425760d06a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:43.411681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f294b6-ff7a-4650-ad23-cb425760d06a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:43.413170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f294b6-ff7a-4650-ad23-cb425760d06a None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:43.428556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-e9f294b6-ff7a-4650-ad23-cb425760d06a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:43.428556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:43.428556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:43.428556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:43.428556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:43.428556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:43.428556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:43.428556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:43.428556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:43.428556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:43.428556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:43.428556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:43.428556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:43.428556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:43.428556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:43.428556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:43.429560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:43.429560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:43.429560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:43.429560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:43.429560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:43.429560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:43.429560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:43.429560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:43.429560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:43.429560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:43.429560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:43.429560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:43.429560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:43.429560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:43.429560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:43.429560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:43.429560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:43.430530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:43.430530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:43.430530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:43.430530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 440/858] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:43.435922 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b15ef-2b43-4f51-8e11-7c2a8f5c4413 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:43.545531 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b15ef-2b43-4f51-8e11-7c2a8f5c4413 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:43.562320 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f294b6-ff7a-4650-ad23-cb425760d06a None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:43.587135 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4011bdd-1a65-4e95-9193-ba71395c6b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:43.587752 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4011bdd-1a65-4e95-9193-ba71395c6b66 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:43.588031 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4011bdd-1a65-4e95-9193-ba71395c6b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:43.588303 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4011bdd-1a65-4e95-9193-ba71395c6b66 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:43.589213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4011bdd-1a65-4e95-9193-ba71395c6b66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:43.589517 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4011bdd-1a65-4e95-9193-ba71395c6b66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:43.591641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4011bdd-1a65-4e95-9193-ba71395c6b66 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:43.611509 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 419/859] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:43.615289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:43.617331 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4011bdd-1a65-4e95-9193-ba71395c6b66 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:43.675965 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:43.681378 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f294b6-ff7a-4650-ad23-cb425760d06a None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:43.717289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f8793-a1ef-4a2a-a3ee-bd47f5d75e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:43.717925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f8793-a1ef-4a2a-a3ee-bd47f5d75e2e None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:43.718231 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f8793-a1ef-4a2a-a3ee-bd47f5d75e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:43.718506 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f8793-a1ef-4a2a-a3ee-bd47f5d75e2e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:43.719085 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f8793-a1ef-4a2a-a3ee-bd47f5d75e2e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:43.721203 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f8793-a1ef-4a2a-a3ee-bd47f5d75e2e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:43.730228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-f33f8793-a1ef-4a2a-a3ee-bd47f5d75e2e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 01:54:43.730228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:43.730228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:54:43.730228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:54:43.730228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:54:43.730228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:54:43.730228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:54:43.730228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:43.730228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 01:54:43.730228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:43.730228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:54:43.730228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:43.730228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:43.730228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:43.730228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:43.730228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:43.731228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:43.731228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:43.731228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:43.731228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:43.731228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:43.731228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:43.731228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:43.731228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:54:43.731228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:54:43.731228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:54:43.731228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:54:43.731228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:43.731228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:43.731228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:43.731228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:43.731228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:43.731228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:43.732146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:43.732146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:43.732146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:43.732146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:43.732146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:43.732146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:43.732146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:43.732146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:43.732146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:43.732146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:43.732146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:54:43.732146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:43.732146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 01:54:43.732146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:43.732146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 441/860] 10.222.0.15 () {56 vars in 1132 bytes} [Wed Sep 14 01:54:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:43.739510 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f8793-a1ef-4a2a-a3ee-bd47f5d75e2e None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:43.742759 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4011bdd-1a65-4e95-9193-ba71395c6b66 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:43.786754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab872c-a457-4b6d-8899-422d3385579e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:43.787485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab872c-a457-4b6d-8899-422d3385579e None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:43.790008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab872c-a457-4b6d-8899-422d3385579e None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:43.790008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab872c-a457-4b6d-8899-422d3385579e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:43.800335 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab872c-a457-4b6d-8899-422d3385579e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:43.802172 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab872c-a457-4b6d-8899-422d3385579e None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:43.818894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-53ab872c-a457-4b6d-8899-422d3385579e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:54:43.818894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:43.818894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:43.818894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:43.818894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:43.818894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:43.818894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:43.818894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:43.818894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:43.818894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:43.818894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:43.818894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:43.818894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:43.818894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:43.818894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:43.818894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:43.820748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:43.820748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:43.820748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:43.820748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:43.820748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:43.820748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:43.820748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:43.820748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:43.820748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:43.820748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:43.820748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:43.820748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:43.820748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:43.820748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:43.820748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:43.820748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:43.820748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:43.820748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:43.822725 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:43.822725 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:43.822725 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:43.822725 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:43.822725 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:43.822725 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:43.822725 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:43.822725 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:43.822725 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:54:43.822725 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:43.822725 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 420/861] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Sep 14 01:54:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:43.829547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab872c-a457-4b6d-8899-422d3385579e None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:43.854243 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f8793-a1ef-4a2a-a3ee-bd47f5d75e2e None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:43.866548 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:43.895071 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66344c2d-9dc1-4719-a465-0e2291028aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:43.895684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66344c2d-9dc1-4719-a465-0e2291028aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:43.895964 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66344c2d-9dc1-4719-a465-0e2291028aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:43.896238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66344c2d-9dc1-4719-a465-0e2291028aa7 None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:43.897023 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66344c2d-9dc1-4719-a465-0e2291028aa7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:43.897330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66344c2d-9dc1-4719-a465-0e2291028aa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:43.898903 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66344c2d-9dc1-4719-a465-0e2291028aa7 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:43.904101 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 442/862] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 14 01:54:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:43.909823 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66344c2d-9dc1-4719-a465-0e2291028aa7 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:43.964730 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab872c-a457-4b6d-8899-422d3385579e None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:43.979930 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:44.011001 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb07f60-73d4-4141-88d2-ea6b1fd9bd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:44.011795 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb07f60-73d4-4141-88d2-ea6b1fd9bd61 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:44.012409 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb07f60-73d4-4141-88d2-ea6b1fd9bd61 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:44.012774 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb07f60-73d4-4141-88d2-ea6b1fd9bd61 None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:44.014078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb07f60-73d4-4141-88d2-ea6b1fd9bd61 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:44.014503 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb07f60-73d4-4141-88d2-ea6b1fd9bd61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:44.016616 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb07f60-73d4-4141-88d2-ea6b1fd9bd61 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:44.023884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66344c2d-9dc1-4719-a465-0e2291028aa7 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:44.046071 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 421/863] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Sep 14 01:54:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:54:44.049298 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb07f60-73d4-4141-88d2-ea6b1fd9bd61 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:44.069799 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee08a0d-5d9e-4343-a2e1-63a5e7411295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:44.070354 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee08a0d-5d9e-4343-a2e1-63a5e7411295 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:44.070845 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee08a0d-5d9e-4343-a2e1-63a5e7411295 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:44.071301 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee08a0d-5d9e-4343-a2e1-63a5e7411295 None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:44.072364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee08a0d-5d9e-4343-a2e1-63a5e7411295 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:44.072858 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee08a0d-5d9e-4343-a2e1-63a5e7411295 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:44.074937 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee08a0d-5d9e-4343-a2e1-63a5e7411295 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:44.085862 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 443/864] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 14 01:54:43 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:44.171946 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb07f60-73d4-4141-88d2-ea6b1fd9bd61 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:44.214030 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a994f-6f1d-4c6c-9bc2-5ab92decbedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:44.214030 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a994f-6f1d-4c6c-9bc2-5ab92decbedc None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:44.214030 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a994f-6f1d-4c6c-9bc2-5ab92decbedc None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:44.214030 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a994f-6f1d-4c6c-9bc2-5ab92decbedc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:44.252725 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:44.291738 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a994f-6f1d-4c6c-9bc2-5ab92decbedc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:44.293415 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a994f-6f1d-4c6c-9bc2-5ab92decbedc None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:44.407781 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 422/865] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:54:44 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:54:44.414015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee08a0d-5d9e-4343-a2e1-63a5e7411295 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:44.455204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:44.518240 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee08a0d-5d9e-4343-a2e1-63a5e7411295 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:44.558789 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04278a5-6db9-47c0-8a43-32befc623e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:44.559877 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04278a5-6db9-47c0-8a43-32befc623e8e None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:44.560157 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04278a5-6db9-47c0-8a43-32befc623e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:44.560430 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04278a5-6db9-47c0-8a43-32befc623e8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:44.571259 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04278a5-6db9-47c0-8a43-32befc623e8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:44.572703 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04278a5-6db9-47c0-8a43-32befc623e8e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:44.587894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-a04278a5-6db9-47c0-8a43-32befc623e8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:44.587894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:44.587894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:44.587894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:44.587894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:44.587894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:44.587894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:44.587894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:44.587894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:44.587894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:44.587894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:44.587894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:44.587894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:44.587894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:44.587894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:44.587894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:44.588967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:44.588967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:44.588967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:44.588967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:44.588967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:44.588967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:44.588967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:44.588967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:44.588967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:44.588967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:44.588967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:44.588967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:44.588967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:44.588967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:44.588967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:44.588967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:44.588967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:44.590006 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:44.590006 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:44.590006 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:44.593589 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 444/866] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:44.595102 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a994f-6f1d-4c6c-9bc2-5ab92decbedc None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:44.664754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:44.698602 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a994f-6f1d-4c6c-9bc2-5ab92decbedc None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:44.733470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd847bcb-ef45-45bb-b631-0893ce1fc002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:44.734133 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd847bcb-ef45-45bb-b631-0893ce1fc002 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:44.734412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd847bcb-ef45-45bb-b631-0893ce1fc002 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:44.734679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd847bcb-ef45-45bb-b631-0893ce1fc002 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:44.735442 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd847bcb-ef45-45bb-b631-0893ce1fc002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:44.735737 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd847bcb-ef45-45bb-b631-0893ce1fc002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:44.737078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd847bcb-ef45-45bb-b631-0893ce1fc002 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:44.748625 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 423/867] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:44.754560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04278a5-6db9-47c0-8a43-32befc623e8e None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:44.848268 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:44.858344 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04278a5-6db9-47c0-8a43-32befc623e8e None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:44.894068 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39950f6-9091-43ae-b786-8136202553b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:44.894705 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39950f6-9091-43ae-b786-8136202553b4 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:44.894984 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39950f6-9091-43ae-b786-8136202553b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:44.895255 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39950f6-9091-43ae-b786-8136202553b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:44.902253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39950f6-9091-43ae-b786-8136202553b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:44.903761 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39950f6-9091-43ae-b786-8136202553b4 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:44.915731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-b39950f6-9091-43ae-b786-8136202553b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:44.915731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:44.915731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:44.915731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:44.915731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:44.915731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:44.915731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:44.915731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:44.915731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:44.915731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:44.915731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:44.915731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:44.915731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:44.915731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:44.915731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:44.916697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:44.916697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:44.916697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:44.916697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:44.916697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:44.916697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:44.916697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:44.916697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:44.916697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:44.916697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:44.916697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:44.916697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:44.916697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:44.916697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:44.916697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:44.916697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:44.916697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:44.917634 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:44.917634 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:44.917634 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:44.917634 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:44.917634 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:44.917634 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:44.917634 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:44.917634 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:44.921741 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 445/868] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:44.926841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd847bcb-ef45-45bb-b631-0893ce1fc002 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:45.029214 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd847bcb-ef45-45bb-b631-0893ce1fc002 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:45.056148 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:45.065000 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406ceb7-696b-456c-9eba-d1195e5753c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:45.065735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406ceb7-696b-456c-9eba-d1195e5753c1 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:45.066036 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406ceb7-696b-456c-9eba-d1195e5753c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:45.066311 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406ceb7-696b-456c-9eba-d1195e5753c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:45.067008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406ceb7-696b-456c-9eba-d1195e5753c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:45.067308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406ceb7-696b-456c-9eba-d1195e5753c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:45.068775 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406ceb7-696b-456c-9eba-d1195e5753c1 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:45.087766 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 424/869] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:45.089156 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39950f6-9091-43ae-b786-8136202553b4 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:45.191842 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39950f6-9091-43ae-b786-8136202553b4 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:45.226148 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f3f585-fd45-4cc3-ad31-93b630114f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:45.226750 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3f585-fd45-4cc3-ad31-93b630114f8d None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:45.227030 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3f585-fd45-4cc3-ad31-93b630114f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:45.227299 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3f585-fd45-4cc3-ad31-93b630114f8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:45.228012 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f3f585-fd45-4cc3-ad31-93b630114f8d None admin] RBAC: Adding query filter params (role.id=e962a2a280224f9fbcd0fdc02262993d, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=e1c8d2eec886482eb135e19ded2c61ae) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:45.228319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f3f585-fd45-4cc3-ad31-93b630114f8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:45.229804 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f3f585-fd45-4cc3-ad31-93b630114f8d None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:45.242238 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 446/870] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:54:45 2022] GET /identity/v3/role_assignments?user.id=e1c8d2eec886482eb135e19ded2c61ae&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=e962a2a280224f9fbcd0fdc02262993d => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:45.377014 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a42d2-94c9-4957-bdc0-b5758944c125 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:45.377476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a42d2-94c9-4957-bdc0-b5758944c125 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:45.377670 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a42d2-94c9-4957-bdc0-b5758944c125 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:45.379399 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 425/871] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:45.387707 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd291d-ec53-4930-b1a2-095984f389a0 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:45.388322 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd291d-ec53-4930-b1a2-095984f389a0 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:45.388815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd291d-ec53-4930-b1a2-095984f389a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:45.414655 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-5fcd291d-ec53-4930-b1a2-095984f389a0 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:45.416677 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-5fcd291d-ec53-4930-b1a2-095984f389a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:45.427764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:45.489018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddea54-32d0-45c9-9fc2-5053ebcfa6c6 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:45.489623 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddea54-32d0-45c9-9fc2-5053ebcfa6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:45.490771 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 447/872] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:54:45.491251 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddea54-32d0-45c9-9fc2-5053ebcfa6c6 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:45.496039 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 426/873] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:45.499582 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a7e96-b4a0-453e-8dbb-f1ec124a4c45 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:45.499702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a7e96-b4a0-453e-8dbb-f1ec124a4c45 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:45.499931 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a7e96-b4a0-453e-8dbb-f1ec124a4c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:45.519466 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-535a7e96-b4a0-453e-8dbb-f1ec124a4c45 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:45.520928 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-535a7e96-b4a0-453e-8dbb-f1ec124a4c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:45.593291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 448/874] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:54:45.770526 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bddea54-32d0-45c9-9fc2-5053ebcfa6c6 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:45.875411 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a7e96-b4a0-453e-8dbb-f1ec124a4c45 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:45.889502 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bddea54-32d0-45c9-9fc2-5053ebcfa6c6 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:45.929068 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3182ab2-afe0-4a6b-a0fd-95089990c7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:45.929744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3182ab2-afe0-4a6b-a0fd-95089990c7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:45.929971 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3182ab2-afe0-4a6b-a0fd-95089990c7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:45.930262 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3182ab2-afe0-4a6b-a0fd-95089990c7d0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:45.930842 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3182ab2-afe0-4a6b-a0fd-95089990c7d0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:45.932215 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3182ab2-afe0-4a6b-a0fd-95089990c7d0 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:45.942687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-a3182ab2-afe0-4a6b-a0fd-95089990c7d0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:54:45.942687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:45.942687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:54:45.942687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:54:45.942687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:54:45.942687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:54:45.942687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:54:45.942687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:45.942687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:54:45.942687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:45.942687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:54:45.942687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:45.942687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:45.942687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:45.942687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:45.942687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:45.942687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:45.943781 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:45.943781 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:45.943781 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:45.943781 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:45.943781 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:45.943781 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:45.943781 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:54:45.943781 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:54:45.943781 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:54:45.943781 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:54:45.943781 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:45.943781 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:45.943781 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:45.943781 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:45.943781 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:45.943781 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:45.944672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:45.944672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:45.944672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:45.944672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:45.944672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:45.944672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:45.944672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:45.944672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:45.944672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:45.944672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:45.944672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:54:45.944672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:45.944672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:54:45.944672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:45.948811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 427/875] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 14 01:54:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:45.949820 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3182ab2-afe0-4a6b-a0fd-95089990c7d0 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:45.977131 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:46.011919 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:46.013129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a7e96-b4a0-453e-8dbb-f1ec124a4c45 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:46.057186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39840a3-f7c3-4898-9254-73570670eebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:46.057976 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39840a3-f7c3-4898-9254-73570670eebf None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:46.058349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39840a3-f7c3-4898-9254-73570670eebf None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:46.058696 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39840a3-f7c3-4898-9254-73570670eebf None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:46.059754 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39840a3-f7c3-4898-9254-73570670eebf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:46.061682 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39840a3-f7c3-4898-9254-73570670eebf None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:46.065201 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3182ab2-afe0-4a6b-a0fd-95089990c7d0 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:46.086044 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 449/876] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 14 01:54:45 2022] POST /identity/v3/services => generated 251 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:54:46.127210 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef97c4-d09d-419e-960b-a4e921b32adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:46.128195 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef97c4-d09d-419e-960b-a4e921b32adb None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:46.128680 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef97c4-d09d-419e-960b-a4e921b32adb None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:46.129160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef97c4-d09d-419e-960b-a4e921b32adb None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:46.130018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef97c4-d09d-419e-960b-a4e921b32adb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:46.130852 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef97c4-d09d-419e-960b-a4e921b32adb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:46.132441 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef97c4-d09d-419e-960b-a4e921b32adb None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:46.138781 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 428/877] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Sep 14 01:54:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:46.147285 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39840a3-f7c3-4898-9254-73570670eebf None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:46.271022 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39840a3-f7c3-4898-9254-73570670eebf None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:46.311466 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a46f3-2447-4412-9a3d-7b16b144747a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:46.312005 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a46f3-2447-4412-9a3d-7b16b144747a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:46.312283 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a46f3-2447-4412-9a3d-7b16b144747a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:46.312553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a46f3-2447-4412-9a3d-7b16b144747a None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:46.313814 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a46f3-2447-4412-9a3d-7b16b144747a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:46.314153 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a46f3-2447-4412-9a3d-7b16b144747a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:46.315517 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a46f3-2447-4412-9a3d-7b16b144747a None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:46.316966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:46.322032 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 450/878] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Sep 14 01:54:46 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:46.371958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10601b-6823-44cd-8ecc-f4c98d9c158d None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:46.372293 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10601b-6823-44cd-8ecc-f4c98d9c158d None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:46.372505 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10601b-6823-44cd-8ecc-f4c98d9c158d None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:46.374157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 429/879] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:46.380640 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde58728-6c41-4973-8755-a5cd37a0b0be None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:46.380971 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde58728-6c41-4973-8755-a5cd37a0b0be None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:46.381195 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde58728-6c41-4973-8755-a5cd37a0b0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:46.404131 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-cde58728-6c41-4973-8755-a5cd37a0b0be None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:46.405337 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-cde58728-6c41-4973-8755-a5cd37a0b0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:46.478628 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 451/880] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:46 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:54:46.750437 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10601b-6823-44cd-8ecc-f4c98d9c158d None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:46.862378 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10601b-6823-44cd-8ecc-f4c98d9c158d None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:46.876445 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:46.900675 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6acd2-04ae-4a3d-892a-a7e6cf68eb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:46.901289 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6acd2-04ae-4a3d-892a-a7e6cf68eb9e None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:46.901568 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6acd2-04ae-4a3d-892a-a7e6cf68eb9e None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:46.901836 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6acd2-04ae-4a3d-892a-a7e6cf68eb9e None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:46.902527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a6acd2-04ae-4a3d-892a-a7e6cf68eb9e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:46.903893 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a6acd2-04ae-4a3d-892a-a7e6cf68eb9e None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:46.923099 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 430/881] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 14 01:54:46 2022] POST /identity/v3/services => generated 269 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:54:47.569306 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31967740-2947-4f0d-b32e-7c347a3d449e None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:47.569845 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31967740-2947-4f0d-b32e-7c347a3d449e None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:47.569845 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31967740-2947-4f0d-b32e-7c347a3d449e None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:47.571433 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 452/882] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:47.576650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7229700-429a-43f6-abd2-6befceec7783 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:47.577091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7229700-429a-43f6-abd2-6befceec7783 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:47.577172 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7229700-429a-43f6-abd2-6befceec7783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:47.595990 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-d7229700-429a-43f6-abd2-6befceec7783 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:47.597172 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-d7229700-429a-43f6-abd2-6befceec7783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:47.671206 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 431/883] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:54:47.932539 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31967740-2947-4f0d-b32e-7c347a3d449e None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:48.034443 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31967740-2947-4f0d-b32e-7c347a3d449e None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:48.070063 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:48.073437 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169904ce-c518-4953-ad97-52f906a64d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:48.074104 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169904ce-c518-4953-ad97-52f906a64d80 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:48.074382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169904ce-c518-4953-ad97-52f906a64d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:48.074657 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169904ce-c518-4953-ad97-52f906a64d80 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:48.085816 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169904ce-c518-4953-ad97-52f906a64d80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:48.090609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-169904ce-c518-4953-ad97-52f906a64d80 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:54:48.090609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:48.090609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:48.090609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:48.090609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:48.090609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:48.090609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:48.090609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:48.090609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:48.090609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:48.090609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:48.090609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:48.090609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:48.090609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:48.090609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:48.090609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:48.091570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:48.091570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:48.091570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:48.091570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:48.091570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:48.091570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:48.091570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:48.091570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:48.091570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:48.091570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:48.091570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:48.091570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:48.091570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:48.091570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:48.091570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:48.091570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:48.092923 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:48.092923 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:48.092923 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:48.092923 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:48.092923 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:48.092923 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:48.092923 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:48.092923 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:48.092923 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:48.092923 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:48.092923 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:54:48.092923 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:48.092923 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 453/884] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Sep 14 01:54:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:48.097759 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7229700-429a-43f6-abd2-6befceec7783 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:48.197680 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:48.202526 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7229700-429a-43f6-abd2-6befceec7783 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:48.238376 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f85ed-7b56-4f92-8906-0743a7ea246b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:48.238981 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f85ed-7b56-4f92-8906-0743a7ea246b None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:48.239253 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f85ed-7b56-4f92-8906-0743a7ea246b None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:48.239519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f85ed-7b56-4f92-8906-0743a7ea246b None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:48.240169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f85ed-7b56-4f92-8906-0743a7ea246b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:48.240463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f85ed-7b56-4f92-8906-0743a7ea246b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:48.241822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f85ed-7b56-4f92-8906-0743a7ea246b None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:48.250426 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 432/885] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Sep 14 01:54:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:48.256990 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169904ce-c518-4953-ad97-52f906a64d80 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:48.363253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169904ce-c518-4953-ad97-52f906a64d80 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:48.408510 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7534e0d-daa5-44ea-8ede-dbcdd96ad562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:48.409122 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7534e0d-daa5-44ea-8ede-dbcdd96ad562 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:48.409397 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7534e0d-daa5-44ea-8ede-dbcdd96ad562 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:48.409662 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7534e0d-daa5-44ea-8ede-dbcdd96ad562 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:48.420269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7534e0d-daa5-44ea-8ede-dbcdd96ad562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:48.421751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7534e0d-daa5-44ea-8ede-dbcdd96ad562 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:48.439085 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-e7534e0d-daa5-44ea-8ede-dbcdd96ad562 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:48.439085 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:48.439085 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:48.439085 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:48.439085 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:48.439085 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:48.439085 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:48.439085 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:48.439085 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:48.439085 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:48.439085 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:48.439085 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:48.439085 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:48.439085 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:48.439085 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:48.439085 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:48.440125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:48.440125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:48.440125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:48.440125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:48.440125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:48.440125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:48.440125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:48.440125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:48.440125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:48.440125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:48.440125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:48.440125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:48.440125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:48.440125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:48.440125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:48.440125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:48.440125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:48.441748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:48.441748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:48.441748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:48.441748 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 454/886] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:48.446549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f85ed-7b56-4f92-8906-0743a7ea246b None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:48.471179 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905be1b-76df-4307-84a3-9664fe6027b4 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:48.471799 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905be1b-76df-4307-84a3-9664fe6027b4 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:48.472313 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905be1b-76df-4307-84a3-9664fe6027b4 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:48.474455 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 455/887] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:54:48.479801 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c79122-f74a-4904-8ed0-fd78b0cf43c5 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:48.480365 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c79122-f74a-4904-8ed0-fd78b0cf43c5 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:48.480847 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c79122-f74a-4904-8ed0-fd78b0cf43c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:48.499648 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-f1c79122-f74a-4904-8ed0-fd78b0cf43c5 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:48.501640 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-f1c79122-f74a-4904-8ed0-fd78b0cf43c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:48.539480 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:48.553944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f85ed-7b56-4f92-8906-0743a7ea246b None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:48.575606 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 456/888] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:54:48.577579 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c517fe0-83a6-4685-8dd6-8ac6281693e0 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:48.578091 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c517fe0-83a6-4685-8dd6-8ac6281693e0 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:48.578574 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c517fe0-83a6-4685-8dd6-8ac6281693e0 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:48.580565 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 457/889] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:48.586023 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ce3b5-ba0d-484c-94ef-b757d7da874c None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:48.586318 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ce3b5-ba0d-484c-94ef-b757d7da874c None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:48.586591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ce3b5-ba0d-484c-94ef-b757d7da874c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:48.594701 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73651ee7-70d2-4dcf-bdc2-1a337f6a3efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:48.595898 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73651ee7-70d2-4dcf-bdc2-1a337f6a3efa None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:48.596422 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73651ee7-70d2-4dcf-bdc2-1a337f6a3efa None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:48.596896 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73651ee7-70d2-4dcf-bdc2-1a337f6a3efa None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:48.597738 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73651ee7-70d2-4dcf-bdc2-1a337f6a3efa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:48.598264 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73651ee7-70d2-4dcf-bdc2-1a337f6a3efa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:48.599817 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73651ee7-70d2-4dcf-bdc2-1a337f6a3efa None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:48.604403 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-c54ce3b5-ba0d-484c-94ef-b757d7da874c None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:48.605574 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-c54ce3b5-ba0d-484c-94ef-b757d7da874c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:48.614021 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 433/890] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:48.626106 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73651ee7-70d2-4dcf-bdc2-1a337f6a3efa None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:48.654313 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:48.682934 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 458/891] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:54:48.740799 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73651ee7-70d2-4dcf-bdc2-1a337f6a3efa None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:48.775594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e983a-763e-4920-b1b9-f7f07ba09274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:48.776462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e983a-763e-4920-b1b9-f7f07ba09274 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:48.776887 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e983a-763e-4920-b1b9-f7f07ba09274 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:48.777328 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e983a-763e-4920-b1b9-f7f07ba09274 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:48.785003 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e983a-763e-4920-b1b9-f7f07ba09274 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:48.786644 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e983a-763e-4920-b1b9-f7f07ba09274 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:48.798892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-463e983a-763e-4920-b1b9-f7f07ba09274 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:54:48.798892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:48.798892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:48.798892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:48.798892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:48.798892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:48.798892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:48.798892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:48.798892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:48.798892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:48.798892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:48.798892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:48.798892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:48.798892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:48.798892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:48.798892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:48.800077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:48.800077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:48.800077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:48.800077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:48.800077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:48.800077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:48.800077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:48.800077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:48.800077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:48.800077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:48.800077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:48.800077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:48.800077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:48.800077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:48.800077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:48.800077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:48.800077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:48.800077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:48.801289 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:48.801289 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:48.801289 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:48.801289 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:48.801289 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:48.801289 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:48.801289 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:48.801289 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:48.801289 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:54:48.801289 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:48.803132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 434/892] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Sep 14 01:54:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:48.808943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ce3b5-ba0d-484c-94ef-b757d7da874c None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:48.863082 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e983a-763e-4920-b1b9-f7f07ba09274 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:48.923778 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:48.927717 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ce3b5-ba0d-484c-94ef-b757d7da874c None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:48.967013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea94d36-2f6f-47d9-bf74-b8b7f327b40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:48.967731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea94d36-2f6f-47d9-bf74-b8b7f327b40c None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:48.968027 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea94d36-2f6f-47d9-bf74-b8b7f327b40c None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:48.968270 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea94d36-2f6f-47d9-bf74-b8b7f327b40c None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:48.968648 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:48.968994 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea94d36-2f6f-47d9-bf74-b8b7f327b40c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:48.969835 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea94d36-2f6f-47d9-bf74-b8b7f327b40c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:48.971328 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea94d36-2f6f-47d9-bf74-b8b7f327b40c None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:48.989576 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 459/893] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Sep 14 01:54:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:48.990237 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea94d36-2f6f-47d9-bf74-b8b7f327b40c None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:48.994464 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e983a-763e-4920-b1b9-f7f07ba09274 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:49.031416 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9868a3c-f554-4397-998b-8e6326784084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:49.032605 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9868a3c-f554-4397-998b-8e6326784084 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:49.032886 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9868a3c-f554-4397-998b-8e6326784084 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:49.033157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9868a3c-f554-4397-998b-8e6326784084 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:49.033749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9868a3c-f554-4397-998b-8e6326784084 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:49.035202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9868a3c-f554-4397-998b-8e6326784084 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:49.045534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-c9868a3c-f554-4397-998b-8e6326784084 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 01:54:49.045534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:49.045534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:54:49.045534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:54:49.045534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:54:49.045534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:54:49.045534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:54:49.045534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:49.045534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 01:54:49.045534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:49.045534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:54:49.045534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:49.045534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:49.045534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:49.045534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:49.045534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:49.045534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:49.050405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:49.050405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:49.050405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:49.050405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:49.050405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:49.050405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:49.050405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:54:49.050405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:54:49.050405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:54:49.050405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:54:49.050405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:49.050405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:49.050405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:49.050405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:49.050405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:49.050405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:49.051294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:49.051294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:49.051294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:49.051294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:49.051294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:49.051294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:49.051294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:49.051294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:49.051294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:49.051294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:49.051294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:54:49.051294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:49.051294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 01:54:49.051294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:49.051294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 435/894] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:54:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:49.052114 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9868a3c-f554-4397-998b-8e6326784084 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:49.104350 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea94d36-2f6f-47d9-bf74-b8b7f327b40c None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:49.143636 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9e2d3-01f7-43ee-b782-79c4c108bccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:49.144249 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9e2d3-01f7-43ee-b782-79c4c108bccd None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:49.144526 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9e2d3-01f7-43ee-b782-79c4c108bccd None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:49.144794 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9e2d3-01f7-43ee-b782-79c4c108bccd None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:49.145441 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d9e2d3-01f7-43ee-b782-79c4c108bccd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:49.146843 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d9e2d3-01f7-43ee-b782-79c4c108bccd None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:49.157348 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9868a3c-f554-4397-998b-8e6326784084 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:49.161729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 460/895] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 14 01:54:48 2022] POST /identity/v3/services => generated 241 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:54:49.162566 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9e2d3-01f7-43ee-b782-79c4c108bccd None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:49.187386 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:49.204897 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f8b72-8707-4279-a3e7-0d233e20b16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:49.205506 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f8b72-8707-4279-a3e7-0d233e20b16f None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:49.205778 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f8b72-8707-4279-a3e7-0d233e20b16f None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:49.206067 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f8b72-8707-4279-a3e7-0d233e20b16f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:49.218400 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:49.280868 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9e2d3-01f7-43ee-b782-79c4c108bccd None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:49.286593 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f8b72-8707-4279-a3e7-0d233e20b16f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:49.288430 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f8b72-8707-4279-a3e7-0d233e20b16f None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:49.318874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6c91f-8156-47ec-aaea-72481deb4d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:49.319960 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6c91f-8156-47ec-aaea-72481deb4d31 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:49.320342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6c91f-8156-47ec-aaea-72481deb4d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:49.320662 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6c91f-8156-47ec-aaea-72481deb4d31 None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:49.321300 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6c91f-8156-47ec-aaea-72481deb4d31 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:49.321603 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6c91f-8156-47ec-aaea-72481deb4d31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:49.323123 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6c91f-8156-47ec-aaea-72481deb4d31 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:49.328205 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 461/896] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:54:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:49.329502 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769e8c0-9911-4c67-8fef-ff6cf4857a8e None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:49.329782 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769e8c0-9911-4c67-8fef-ff6cf4857a8e None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:49.330084 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769e8c0-9911-4c67-8fef-ff6cf4857a8e None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:49.331412 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 462/897] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:49.335492 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769e8c0-9911-4c67-8fef-ff6cf4857a8e None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:49.407070 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 436/898] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:54:49 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:54:49.408325 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19524a-16d1-46d4-99ef-8d14c2962ce1 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:49.408607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19524a-16d1-46d4-99ef-8d14c2962ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:49.408876 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19524a-16d1-46d4-99ef-8d14c2962ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:49.426710 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-fd19524a-16d1-46d4-99ef-8d14c2962ce1 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:49.427874 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-fd19524a-16d1-46d4-99ef-8d14c2962ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:49.431392 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:49.451896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769e8c0-9911-4c67-8fef-ff6cf4857a8e None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:49.464126 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:49.489912 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c37b0e-2b3e-49d9-8899-a1eeb7449456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:49.490555 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c37b0e-2b3e-49d9-8899-a1eeb7449456 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:49.490832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c37b0e-2b3e-49d9-8899-a1eeb7449456 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:49.491110 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c37b0e-2b3e-49d9-8899-a1eeb7449456 None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:49.491727 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c37b0e-2b3e-49d9-8899-a1eeb7449456 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:49.492030 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c37b0e-2b3e-49d9-8899-a1eeb7449456 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:49.493401 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c37b0e-2b3e-49d9-8899-a1eeb7449456 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:49.497701 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 437/899] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:54:49.498885 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19524a-16d1-46d4-99ef-8d14c2962ce1 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:49.503597 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 463/900] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:54:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:54:49.504373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c37b0e-2b3e-49d9-8899-a1eeb7449456 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:49.614896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c37b0e-2b3e-49d9-8899-a1eeb7449456 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:49.637056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19524a-16d1-46d4-99ef-8d14c2962ce1 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:49.655843 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74991cd6-8be1-4fcc-8df7-5be3a84252b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:49.656465 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74991cd6-8be1-4fcc-8df7-5be3a84252b7 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:49.656771 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74991cd6-8be1-4fcc-8df7-5be3a84252b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:49.657073 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74991cd6-8be1-4fcc-8df7-5be3a84252b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:49.657798 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74991cd6-8be1-4fcc-8df7-5be3a84252b7 None admin] RBAC: Adding query filter params (interface=public, service_id=a7eb6741bdfd425d8dad4559a8367aa8, region_id=RegionOne) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:49.658139 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74991cd6-8be1-4fcc-8df7-5be3a84252b7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:49.659831 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74991cd6-8be1-4fcc-8df7-5be3a84252b7 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:49.670027 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 464/901] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Sep 14 01:54:49 2022] GET /identity/v3/endpoints?service_id=a7eb6741bdfd425d8dad4559a8367aa8&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:49.673490 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74991cd6-8be1-4fcc-8df7-5be3a84252b7 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:49.690445 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940ce80-b0c5-4fd3-bfd2-5756cdc198b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:49.691403 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:49.692162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940ce80-b0c5-4fd3-bfd2-5756cdc198b2 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:49.692442 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940ce80-b0c5-4fd3-bfd2-5756cdc198b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:49.692716 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940ce80-b0c5-4fd3-bfd2-5756cdc198b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:49.706988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0940ce80-b0c5-4fd3-bfd2-5756cdc198b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:49.709170 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0940ce80-b0c5-4fd3-bfd2-5756cdc198b2 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:49.731078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-0940ce80-b0c5-4fd3-bfd2-5756cdc198b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:49.731078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:49.731078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:49.731078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:49.731078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:49.731078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:49.731078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:49.731078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:49.731078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:49.731078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:49.731078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:49.731078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:49.731078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:49.731078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:49.731078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:49.731078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:49.732097 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:49.732097 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:49.732097 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:49.732097 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:49.732097 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:49.732097 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:49.732097 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:49.732097 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:49.732097 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:49.732097 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:49.732097 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:49.732097 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:49.732097 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:49.732097 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:49.732097 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:49.732097 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:49.732097 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:49.733032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:49.733032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:49.733032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:49.735157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 438/902] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:49.743062 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940ce80-b0c5-4fd3-bfd2-5756cdc198b2 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:49.773878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:49.796185 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74991cd6-8be1-4fcc-8df7-5be3a84252b7 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:49.833032 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a2973-9673-4fd9-a972-75cad44401d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:49.833497 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a2973-9673-4fd9-a972-75cad44401d9 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:49.833775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a2973-9673-4fd9-a972-75cad44401d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:49.834136 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a2973-9673-4fd9-a972-75cad44401d9 None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:49.834811 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a2973-9673-4fd9-a972-75cad44401d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:49.836013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a2973-9673-4fd9-a972-75cad44401d9 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:49.842110 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 465/903] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Sep 14 01:54:49 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:54:49.842733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a2973-9673-4fd9-a972-75cad44401d9 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:49.868964 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940ce80-b0c5-4fd3-bfd2-5756cdc198b2 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:49.910545 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409bd9e-8e7f-4b71-a5ee-d080f15db47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:49.911313 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409bd9e-8e7f-4b71-a5ee-d080f15db47a None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:49.911691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409bd9e-8e7f-4b71-a5ee-d080f15db47a None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:49.912072 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409bd9e-8e7f-4b71-a5ee-d080f15db47a None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:49.912846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409bd9e-8e7f-4b71-a5ee-d080f15db47a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:49.913250 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409bd9e-8e7f-4b71-a5ee-d080f15db47a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:49.914781 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409bd9e-8e7f-4b71-a5ee-d080f15db47a None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:49.932891 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 439/904] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:49.939020 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409bd9e-8e7f-4b71-a5ee-d080f15db47a None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:49.945774 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:49.975964 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a2973-9673-4fd9-a972-75cad44401d9 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:50.016742 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea6e7d-80c0-4834-a83f-ad708dd4c371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:50.017546 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea6e7d-80c0-4834-a83f-ad708dd4c371 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:50.018228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea6e7d-80c0-4834-a83f-ad708dd4c371 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:50.018733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea6e7d-80c0-4834-a83f-ad708dd4c371 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:50.019471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ea6e7d-80c0-4834-a83f-ad708dd4c371 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:50.021702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ea6e7d-80c0-4834-a83f-ad708dd4c371 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:50.032687 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-53ea6e7d-80c0-4834-a83f-ad708dd4c371 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 01:54:50.032687 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:50.032687 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:54:50.032687 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:54:50.032687 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:54:50.032687 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:54:50.032687 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:54:50.032687 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:50.032687 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 01:54:50.032687 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:50.032687 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:54:50.032687 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:50.032687 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:50.032687 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:50.032687 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:50.032687 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:50.033874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:50.033874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:50.033874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:50.033874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:50.033874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:50.033874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:50.033874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:50.033874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:54:50.033874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:54:50.033874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:54:50.033874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:54:50.033874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:50.033874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:50.033874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:50.033874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:50.033874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:50.033874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:50.034948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:50.034948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:50.034948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:50.034948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:50.034948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:50.034948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:50.034948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:50.034948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:50.034948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:50.034948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:50.034948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:54:50.034948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:50.034948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 01:54:50.034948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:50.034948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 466/905] 10.222.0.15 () {56 vars in 1132 bytes} [Wed Sep 14 01:54:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:50.042469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea6e7d-80c0-4834-a83f-ad708dd4c371 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:50.061476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:50.082771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409bd9e-8e7f-4b71-a5ee-d080f15db47a None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:50.117633 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762498f2-152b-4d63-926e-b2fc56b4aeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:50.118472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762498f2-152b-4d63-926e-b2fc56b4aeb0 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:50.118830 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762498f2-152b-4d63-926e-b2fc56b4aeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:50.119251 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762498f2-152b-4d63-926e-b2fc56b4aeb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:50.126525 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762498f2-152b-4d63-926e-b2fc56b4aeb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:50.127761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762498f2-152b-4d63-926e-b2fc56b4aeb0 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:50.139352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-762498f2-152b-4d63-926e-b2fc56b4aeb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:50.139352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:50.139352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:50.139352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:50.139352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:50.139352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:50.139352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:50.139352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:50.139352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:50.139352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:50.139352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:50.139352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:50.139352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:50.139352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:50.139352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:50.140295 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:50.140295 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:50.140295 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:50.140295 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:50.140295 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:50.140295 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:50.140295 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:50.140295 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:50.140295 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:50.140295 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:50.140295 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:50.140295 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:50.140295 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:50.140295 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:50.140295 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:50.140295 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:50.140295 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:50.141804 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:50.141804 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:50.141804 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:50.141804 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:50.141804 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:50.141804 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:50.141804 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:50.141804 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:50.141804 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 440/906] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:50.148662 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762498f2-152b-4d63-926e-b2fc56b4aeb0 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:50.161705 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea6e7d-80c0-4834-a83f-ad708dd4c371 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:50.206023 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5664d99-e575-4d3f-a897-9ee62e047ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:50.207223 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5664d99-e575-4d3f-a897-9ee62e047ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:50.207701 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5664d99-e575-4d3f-a897-9ee62e047ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:50.208313 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5664d99-e575-4d3f-a897-9ee62e047ab1 None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:50.209442 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5664d99-e575-4d3f-a897-9ee62e047ab1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:50.210191 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5664d99-e575-4d3f-a897-9ee62e047ab1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:50.212203 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5664d99-e575-4d3f-a897-9ee62e047ab1 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:50.218582 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 467/907] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 14 01:54:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:54:50.224242 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5664d99-e575-4d3f-a897-9ee62e047ab1 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:50.227383 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:50.265031 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762498f2-152b-4d63-926e-b2fc56b4aeb0 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:50.301899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1d502-ccc7-4542-b12f-3ef91d682323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:50.302503 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1d502-ccc7-4542-b12f-3ef91d682323 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:50.302778 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1d502-ccc7-4542-b12f-3ef91d682323 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:50.303045 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1d502-ccc7-4542-b12f-3ef91d682323 None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:50.303725 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1d502-ccc7-4542-b12f-3ef91d682323 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:50.304084 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1d502-ccc7-4542-b12f-3ef91d682323 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:50.305511 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1d502-ccc7-4542-b12f-3ef91d682323 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:50.322346 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 441/908] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:54:50.324715 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1d502-ccc7-4542-b12f-3ef91d682323 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:50.327490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:50.343429 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5664d99-e575-4d3f-a897-9ee62e047ab1 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:50.382515 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48102c7c-dfb9-4ad3-9fad-0b25f2f32add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:50.382515 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48102c7c-dfb9-4ad3-9fad-0b25f2f32add None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:50.382515 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48102c7c-dfb9-4ad3-9fad-0b25f2f32add None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:50.382515 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48102c7c-dfb9-4ad3-9fad-0b25f2f32add None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:50.382959 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48102c7c-dfb9-4ad3-9fad-0b25f2f32add None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:50.382959 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48102c7c-dfb9-4ad3-9fad-0b25f2f32add None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:50.384190 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48102c7c-dfb9-4ad3-9fad-0b25f2f32add None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:50.392895 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 468/909] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 14 01:54:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:54:50.398979 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48102c7c-dfb9-4ad3-9fad-0b25f2f32add None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:50.447217 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1d502-ccc7-4542-b12f-3ef91d682323 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:50.470233 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:50.484219 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec45ce-2ee8-4a60-9014-aa6a1c85f01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:50.485238 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec45ce-2ee8-4a60-9014-aa6a1c85f01a None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:50.485618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec45ce-2ee8-4a60-9014-aa6a1c85f01a None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:50.485895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec45ce-2ee8-4a60-9014-aa6a1c85f01a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:50.486797 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec45ce-2ee8-4a60-9014-aa6a1c85f01a None admin] RBAC: Adding query filter params (role.id=4f2c7f12acdc45b48a2271b06f69a600, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=e1c8d2eec886482eb135e19ded2c61ae) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:50.487125 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec45ce-2ee8-4a60-9014-aa6a1c85f01a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:50.488736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec45ce-2ee8-4a60-9014-aa6a1c85f01a None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:50.498265 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 442/910] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:54:50 2022] GET /identity/v3/role_assignments?user.id=e1c8d2eec886482eb135e19ded2c61ae&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=4f2c7f12acdc45b48a2271b06f69a600 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:54:50.507693 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48102c7c-dfb9-4ad3-9fad-0b25f2f32add None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:50.546867 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617ec87-9c14-415c-80b5-e3adf45fe3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:50.547664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617ec87-9c14-415c-80b5-e3adf45fe3f3 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:50.548022 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617ec87-9c14-415c-80b5-e3adf45fe3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:50.548359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617ec87-9c14-415c-80b5-e3adf45fe3f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:50.549149 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617ec87-9c14-415c-80b5-e3adf45fe3f3 None admin] RBAC: Adding query filter params (interface=public, service_id=492ef4a47ffa477dae5938e72b23cec3, region_id=RegionOne) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:50.549523 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617ec87-9c14-415c-80b5-e3adf45fe3f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:50.551051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617ec87-9c14-415c-80b5-e3adf45fe3f3 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:50.557425 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 469/911] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Sep 14 01:54:50 2022] GET /identity/v3/endpoints?service_id=492ef4a47ffa477dae5938e72b23cec3&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:50.563756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec45ce-2ee8-4a60-9014-aa6a1c85f01a None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:50.637780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:50.679685 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec45ce-2ee8-4a60-9014-aa6a1c85f01a None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:50.715975 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591169b1-71d0-4888-96bb-779405e4aca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:50.716576 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591169b1-71d0-4888-96bb-779405e4aca2 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:50.716850 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591169b1-71d0-4888-96bb-779405e4aca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:50.717121 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591169b1-71d0-4888-96bb-779405e4aca2 None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:50.717714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591169b1-71d0-4888-96bb-779405e4aca2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:50.719671 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591169b1-71d0-4888-96bb-779405e4aca2 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:50.726052 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 443/912] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Sep 14 01:54:50 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:54:51.350532 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df7001-a078-406d-ba95-2b4b9516f8c7 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:51.351502 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df7001-a078-406d-ba95-2b4b9516f8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:51.352008 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df7001-a078-406d-ba95-2b4b9516f8c7 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:51.353815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 470/913] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:51.359352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a566d4-054e-4d0c-a9a9-00e2144abfc4 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:51.359898 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a566d4-054e-4d0c-a9a9-00e2144abfc4 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:51.360392 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a566d4-054e-4d0c-a9a9-00e2144abfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:51.380726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-c0a566d4-054e-4d0c-a9a9-00e2144abfc4 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:51.382294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-c0a566d4-054e-4d0c-a9a9-00e2144abfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:51.454861 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 444/914] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:54:51.728619 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df7001-a078-406d-ba95-2b4b9516f8c7 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:51.837651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df7001-a078-406d-ba95-2b4b9516f8c7 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:51.867294 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:51.873408 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2d948-360a-4584-8aac-9ce619edb138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:51.874110 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2d948-360a-4584-8aac-9ce619edb138 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:51.874577 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2d948-360a-4584-8aac-9ce619edb138 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:51.875057 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2d948-360a-4584-8aac-9ce619edb138 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:51.875698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2d948-360a-4584-8aac-9ce619edb138 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:51.877069 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2d948-360a-4584-8aac-9ce619edb138 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:51.886784 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-6ae2d948-360a-4584-8aac-9ce619edb138 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:54:51.886784 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:51.886784 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:54:51.886784 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:54:51.886784 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:54:51.886784 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:54:51.886784 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:54:51.886784 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:51.886784 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:54:51.886784 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:51.886784 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:54:51.886784 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:51.886784 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:51.886784 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:51.886784 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:51.886784 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:51.886784 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:51.887791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:51.887791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:51.887791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:51.887791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:51.887791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:51.887791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:51.887791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:54:51.887791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:54:51.887791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:54:51.887791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:54:51.887791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:51.887791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:51.887791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:51.887791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:51.887791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:51.887791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:51.888671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:51.888671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:51.888671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:51.888671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:51.888671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:51.888671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:51.888671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:51.888671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:51.888671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:51.888671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:51.888671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:54:51.888671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:51.888671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:54:51.888671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:51.888671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 471/915] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 14 01:54:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:51.895366 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a566d4-054e-4d0c-a9a9-00e2144abfc4 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:51.960246 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:52.002552 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a566d4-054e-4d0c-a9a9-00e2144abfc4 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:52.042519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cf54e-4d04-4732-972c-746a02fcc1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:52.043150 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cf54e-4d04-4732-972c-746a02fcc1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:52.043427 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cf54e-4d04-4732-972c-746a02fcc1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:52.043699 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cf54e-4d04-4732-972c-746a02fcc1a0 None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:52.044349 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7cf54e-4d04-4732-972c-746a02fcc1a0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:52.044651 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7cf54e-4d04-4732-972c-746a02fcc1a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:52.046175 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7cf54e-4d04-4732-972c-746a02fcc1a0 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:52.052142 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 445/916] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Sep 14 01:54:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:52.059176 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2d948-360a-4584-8aac-9ce619edb138 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:52.147561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d70b5-124c-4a21-a39e-7536694be819 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:52.147885 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d70b5-124c-4a21-a39e-7536694be819 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:52.148109 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d70b5-124c-4a21-a39e-7536694be819 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:52.149612 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 446/917] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:54:52.154838 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d0f6d-63e4-4aee-af90-9d93455e5c1c None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:52.155344 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d0f6d-63e4-4aee-af90-9d93455e5c1c None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:52.155811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d0f6d-63e4-4aee-af90-9d93455e5c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:52.173518 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2d948-360a-4584-8aac-9ce619edb138 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:52.175680 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-048d0f6d-63e4-4aee-af90-9d93455e5c1c None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:52.177057 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-048d0f6d-63e4-4aee-af90-9d93455e5c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:52.216237 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e7320-3597-4645-a677-2056f3e6804e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:52.216848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e7320-3597-4645-a677-2056f3e6804e None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:52.217126 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e7320-3597-4645-a677-2056f3e6804e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:52.217400 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e7320-3597-4645-a677-2056f3e6804e None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:52.218043 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e7320-3597-4645-a677-2056f3e6804e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:52.218350 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e7320-3597-4645-a677-2056f3e6804e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:52.220251 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e7320-3597-4645-a677-2056f3e6804e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:52.226960 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 472/918] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Sep 14 01:54:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:52.232809 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e7320-3597-4645-a677-2056f3e6804e None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:52.254183 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 447/919] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:54:52.318379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:52.338917 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e7320-3597-4645-a677-2056f3e6804e None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:52.373582 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54feb620-3171-42ff-a174-200be2e4381e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:52.374271 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54feb620-3171-42ff-a174-200be2e4381e None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:52.374484 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54feb620-3171-42ff-a174-200be2e4381e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:52.374761 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54feb620-3171-42ff-a174-200be2e4381e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:52.375514 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54feb620-3171-42ff-a174-200be2e4381e None admin] RBAC: Adding query filter params (interface=public, service_id=67b6d7123d314ff39290b96a553be85e, region_id=RegionOne) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:52.375825 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54feb620-3171-42ff-a174-200be2e4381e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:52.377193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54feb620-3171-42ff-a174-200be2e4381e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:52.383004 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 473/920] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Sep 14 01:54:52 2022] GET /identity/v3/endpoints?service_id=67b6d7123d314ff39290b96a553be85e&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:52.389030 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d0f6d-63e4-4aee-af90-9d93455e5c1c None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:52.450588 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:52.497176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d0f6d-63e4-4aee-af90-9d93455e5c1c None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:52.527497 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54feb620-3171-42ff-a174-200be2e4381e None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:52.533173 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506dc25-14d2-4bad-9ef2-87cd854fa5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:52.533790 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506dc25-14d2-4bad-9ef2-87cd854fa5fe None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:52.534095 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506dc25-14d2-4bad-9ef2-87cd854fa5fe None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:52.534371 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506dc25-14d2-4bad-9ef2-87cd854fa5fe None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:52.534975 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d506dc25-14d2-4bad-9ef2-87cd854fa5fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:52.536336 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d506dc25-14d2-4bad-9ef2-87cd854fa5fe None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:52.542722 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 448/921] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Sep 14 01:54:52 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:54:52.647669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54feb620-3171-42ff-a174-200be2e4381e None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:52.673240 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:52.686183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cded376-da9f-4703-8d5e-6e20c230c50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:52.687104 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cded376-da9f-4703-8d5e-6e20c230c50b None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:52.687600 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cded376-da9f-4703-8d5e-6e20c230c50b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:52.688099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cded376-da9f-4703-8d5e-6e20c230c50b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:52.688832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cded376-da9f-4703-8d5e-6e20c230c50b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:52.690439 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cded376-da9f-4703-8d5e-6e20c230c50b None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:52.700246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-2cded376-da9f-4703-8d5e-6e20c230c50b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 01:54:52.700246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:52.700246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:54:52.700246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:54:52.700246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:54:52.700246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:54:52.700246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:54:52.700246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:52.700246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 01:54:52.700246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:52.700246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:54:52.700246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:52.700246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:52.700246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:52.700246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:52.700246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:52.700246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:52.701530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:52.701530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:52.701530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:52.701530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:52.701530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:52.701530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:52.701530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:54:52.701530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:54:52.701530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:54:52.701530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:54:52.701530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:52.701530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:52.701530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:52.701530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:52.701530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:52.701530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:52.702607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:52.702607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:52.702607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:52.702607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:52.702607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:52.702607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:52.702607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:52.702607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:52.702607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:52.702607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:52.702607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:54:52.702607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:52.702607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 01:54:52.702607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:52.710110 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 474/922] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:54:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:52.711870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506dc25-14d2-4bad-9ef2-87cd854fa5fe None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:52.724220 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae49bdd-7e92-4d4b-ac47-46be2f004310 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:52.724560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae49bdd-7e92-4d4b-ac47-46be2f004310 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:52.724762 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae49bdd-7e92-4d4b-ac47-46be2f004310 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:52.726419 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 475/923] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:54:52.731152 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9811b-c1ce-480e-ba3c-6ddd0db98fe6 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:52.731442 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9811b-c1ce-480e-ba3c-6ddd0db98fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:52.731713 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9811b-c1ce-480e-ba3c-6ddd0db98fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:52.750117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-dae9811b-c1ce-480e-ba3c-6ddd0db98fe6 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:52.751846 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-dae9811b-c1ce-480e-ba3c-6ddd0db98fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:52.834310 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506dc25-14d2-4bad-9ef2-87cd854fa5fe None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:52.839289 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 476/924] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:54:52.862935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:52.882506 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5cc95f-cebf-4b75-8b13-5a64008887b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:52.883111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cc95f-cebf-4b75-8b13-5a64008887b3 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:52.883389 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cc95f-cebf-4b75-8b13-5a64008887b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:52.883652 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cc95f-cebf-4b75-8b13-5a64008887b3 None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:52.884269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5cc95f-cebf-4b75-8b13-5a64008887b3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:52.884560 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5cc95f-cebf-4b75-8b13-5a64008887b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:52.885913 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5cc95f-cebf-4b75-8b13-5a64008887b3 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:52.891643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 449/925] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:54:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:52.897109 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9811b-c1ce-480e-ba3c-6ddd0db98fe6 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:52.951105 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9dbdc3-ed7e-48e2-9549-dd7d972fbee8 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:52.951637 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9dbdc3-ed7e-48e2-9549-dd7d972fbee8 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:52.951637 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9dbdc3-ed7e-48e2-9549-dd7d972fbee8 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:52.952952 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 450/926] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:52.958945 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae0fe6-2aec-43ff-a835-ad297479398d None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:52.959536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae0fe6-2aec-43ff-a835-ad297479398d None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:52.959990 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae0fe6-2aec-43ff-a835-ad297479398d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:52.980479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-8eae0fe6-2aec-43ff-a835-ad297479398d None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:52.982014 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-8eae0fe6-2aec-43ff-a835-ad297479398d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:53.007878 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9811b-c1ce-480e-ba3c-6ddd0db98fe6 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:53.028570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:53.051702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f7547-f33c-4490-a870-3de00c497c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:53.052439 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f7547-f33c-4490-a870-3de00c497c71 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:53.052830 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f7547-f33c-4490-a870-3de00c497c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:53.053123 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f7547-f33c-4490-a870-3de00c497c71 None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:53.053747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f7547-f33c-4490-a870-3de00c497c71 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:53.054080 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f7547-f33c-4490-a870-3de00c497c71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:53.056093 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f7547-f33c-4490-a870-3de00c497c71 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:53.066013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 477/927] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 14 01:54:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:53.066637 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 451/928] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:54:53.067785 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f7547-f33c-4490-a870-3de00c497c71 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:53.123235 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae0fe6-2aec-43ff-a835-ad297479398d None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:53.170982 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f7547-f33c-4490-a870-3de00c497c71 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:53.207948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93881aa3-a29c-4cb6-bf83-8ee71c576b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:53.208698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93881aa3-a29c-4cb6-bf83-8ee71c576b4e None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:53.209155 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93881aa3-a29c-4cb6-bf83-8ee71c576b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:53.209634 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93881aa3-a29c-4cb6-bf83-8ee71c576b4e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:53.210441 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93881aa3-a29c-4cb6-bf83-8ee71c576b4e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:53.211910 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93881aa3-a29c-4cb6-bf83-8ee71c576b4e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:53.234995 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 478/929] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 14 01:54:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:54:53.264337 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae0fe6-2aec-43ff-a835-ad297479398d None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:53.280959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:53.311110 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511f807-e799-4b58-aecd-24beb197f01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:53.311753 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511f807-e799-4b58-aecd-24beb197f01e None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:53.312037 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511f807-e799-4b58-aecd-24beb197f01e None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:53.312308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511f807-e799-4b58-aecd-24beb197f01e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:53.324687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7511f807-e799-4b58-aecd-24beb197f01e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:53.330618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-7511f807-e799-4b58-aecd-24beb197f01e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:54:53.330618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:53.330618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:53.330618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:53.330618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:53.330618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:53.330618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:53.330618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:53.330618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:53.330618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:53.330618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:53.330618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:53.330618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:53.330618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:53.330618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:53.330618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:53.331656 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:53.331656 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:53.331656 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:53.331656 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:53.331656 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:53.331656 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:53.331656 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:53.331656 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:53.331656 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:53.331656 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:53.331656 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:53.331656 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:53.331656 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:53.331656 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:53.331656 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:53.331656 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:53.332520 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:53.332520 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:53.332520 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:53.332520 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:53.332520 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:53.332520 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:53.332520 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:53.332520 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:53.332520 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:53.332520 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:53.332520 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:54:53.332520 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:53.332520 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 452/930] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Sep 14 01:54:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:53.336105 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93881aa3-a29c-4cb6-bf83-8ee71c576b4e None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:53.340338 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511f807-e799-4b58-aecd-24beb197f01e None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:53.389793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:53.457709 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511f807-e799-4b58-aecd-24beb197f01e None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:53.473619 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93881aa3-a29c-4cb6-bf83-8ee71c576b4e None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:53.498454 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8a540-e5a1-4771-83ce-ceddd0466ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:53.499462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8a540-e5a1-4771-83ce-ceddd0466ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:53.499948 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8a540-e5a1-4771-83ce-ceddd0466ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:53.500994 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8a540-e5a1-4771-83ce-ceddd0466ed4 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:53.501829 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d8a540-e5a1-4771-83ce-ceddd0466ed4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:53.502330 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d8a540-e5a1-4771-83ce-ceddd0466ed4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:53.503695 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d8a540-e5a1-4771-83ce-ceddd0466ed4 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:53.512232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 453/931] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Sep 14 01:54:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:53.518592 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8a540-e5a1-4771-83ce-ceddd0466ed4 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:53.524375 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ea113-9dad-48d5-a1cb-23809db55f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:53.525099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ea113-9dad-48d5-a1cb-23809db55f5d None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:53.525384 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ea113-9dad-48d5-a1cb-23809db55f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:53.525652 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ea113-9dad-48d5-a1cb-23809db55f5d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:53.526777 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ea113-9dad-48d5-a1cb-23809db55f5d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:53.527798 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ea113-9dad-48d5-a1cb-23809db55f5d None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:53.540592 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-146ea113-9dad-48d5-a1cb-23809db55f5d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 01:54:53.540592 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:53.540592 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:54:53.540592 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:54:53.540592 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:54:53.540592 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:54:53.540592 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:54:53.540592 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:53.540592 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 01:54:53.540592 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:53.540592 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:54:53.540592 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:53.540592 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:53.540592 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:53.540592 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:53.540592 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:53.541675 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:53.541675 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:53.541675 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:53.541675 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:53.541675 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:53.541675 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:53.541675 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:53.541675 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:54:53.541675 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:54:53.541675 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:54:53.541675 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:54:53.541675 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:53.541675 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:53.541675 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:53.541675 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:53.541675 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:53.541675 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:53.543082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:53.543082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:53.543082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:53.543082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:53.543082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:53.543082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:53.543082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:53.543082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:53.543082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:53.543082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:53.543082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:54:53.543082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:53.543082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 01:54:53.543082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:53.543082 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 479/932] 10.222.0.15 () {56 vars in 1132 bytes} [Wed Sep 14 01:54:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:53.547960 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ea113-9dad-48d5-a1cb-23809db55f5d None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:53.559893 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:53.640675 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8a540-e5a1-4771-83ce-ceddd0466ed4 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:53.654607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:53.657886 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ea113-9dad-48d5-a1cb-23809db55f5d None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:53.679465 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2658c12-22be-43f3-afaa-eefed89ef0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:53.680481 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2658c12-22be-43f3-afaa-eefed89ef0bb None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:53.681028 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2658c12-22be-43f3-afaa-eefed89ef0bb None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:53.681483 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2658c12-22be-43f3-afaa-eefed89ef0bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:53.694206 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2658c12-22be-43f3-afaa-eefed89ef0bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:53.695635 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e17e76-bf64-4a8a-b3e6-f2de0986ff2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:53.696141 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2658c12-22be-43f3-afaa-eefed89ef0bb None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:53.696251 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e17e76-bf64-4a8a-b3e6-f2de0986ff2b None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:53.696513 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e17e76-bf64-4a8a-b3e6-f2de0986ff2b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:53.696785 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e17e76-bf64-4a8a-b3e6-f2de0986ff2b None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:53.697592 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e17e76-bf64-4a8a-b3e6-f2de0986ff2b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:53.697893 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e17e76-bf64-4a8a-b3e6-f2de0986ff2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:53.699392 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e17e76-bf64-4a8a-b3e6-f2de0986ff2b None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:53.704764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 480/933] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 14 01:54:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:54:53.709947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e17e76-bf64-4a8a-b3e6-f2de0986ff2b None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:53.713438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-a2658c12-22be-43f3-afaa-eefed89ef0bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:53.713438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:53.713438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:53.713438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:53.713438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:53.713438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:53.713438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:53.713438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:53.713438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:53.713438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:53.713438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:53.713438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:53.713438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:53.713438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:53.713438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:53.713438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:53.714469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:53.714469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:53.714469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:53.714469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:53.714469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:53.714469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:53.714469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:53.714469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:53.714469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:53.714469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:53.714469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:53.714469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:53.714469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:53.714469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:53.714469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:53.714469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:53.714469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:53.715409 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:53.715409 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:53.715409 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:53.722129 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 454/934] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:53.724322 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2658c12-22be-43f3-afaa-eefed89ef0bb None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:53.824493 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e17e76-bf64-4a8a-b3e6-f2de0986ff2b None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:53.828884 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:53.848747 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2658c12-22be-43f3-afaa-eefed89ef0bb None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:53.866575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016acfc4-d43c-4690-9287-9f600039b1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:53.867581 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016acfc4-d43c-4690-9287-9f600039b1fb None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:53.868140 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016acfc4-d43c-4690-9287-9f600039b1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:53.868591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016acfc4-d43c-4690-9287-9f600039b1fb None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:53.869421 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016acfc4-d43c-4690-9287-9f600039b1fb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:53.869903 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016acfc4-d43c-4690-9287-9f600039b1fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:53.871669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016acfc4-d43c-4690-9287-9f600039b1fb None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:53.877594 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 481/935] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 14 01:54:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:53.886022 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016acfc4-d43c-4690-9287-9f600039b1fb None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:53.889685 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763afe67-7c17-4abc-bb93-d9cdb11a3fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:53.890532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763afe67-7c17-4abc-bb93-d9cdb11a3fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:53.891413 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763afe67-7c17-4abc-bb93-d9cdb11a3fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:53.891704 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763afe67-7c17-4abc-bb93-d9cdb11a3fb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:53.892405 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763afe67-7c17-4abc-bb93-d9cdb11a3fb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:53.892707 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763afe67-7c17-4abc-bb93-d9cdb11a3fb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:53.894111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763afe67-7c17-4abc-bb93-d9cdb11a3fb4 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:53.912266 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 455/936] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:54:53.913358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763afe67-7c17-4abc-bb93-d9cdb11a3fb4 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:53.919192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:53.996567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016acfc4-d43c-4690-9287-9f600039b1fb None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:54.021352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763afe67-7c17-4abc-bb93-d9cdb11a3fb4 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:54.041872 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec9f8d-c37b-4825-9b9f-6ba59aafd81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:54.042747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec9f8d-c37b-4825-9b9f-6ba59aafd81f None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:54.043158 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec9f8d-c37b-4825-9b9f-6ba59aafd81f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:54.043526 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec9f8d-c37b-4825-9b9f-6ba59aafd81f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:54.044724 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec9f8d-c37b-4825-9b9f-6ba59aafd81f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:54.046533 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec9f8d-c37b-4825-9b9f-6ba59aafd81f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:54.059734 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a46e57-956d-433c-a2ec-4507b64e25f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:54.060472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a46e57-956d-433c-a2ec-4507b64e25f8 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:54.061037 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a46e57-956d-433c-a2ec-4507b64e25f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:54.061272 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a46e57-956d-433c-a2ec-4507b64e25f8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:54.069789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a46e57-956d-433c-a2ec-4507b64e25f8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:54.071465 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a46e57-956d-433c-a2ec-4507b64e25f8 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:54.072136 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 482/937] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 14 01:54:53 2022] POST /identity/v3/endpoints => generated 347 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:54:54.085076 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-b4a46e57-956d-433c-a2ec-4507b64e25f8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:54:54.085076 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:54.085076 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:54.085076 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:54.085076 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:54.085076 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:54.085076 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:54.085076 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:54.085076 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:54.085076 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:54.085076 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:54.085076 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:54.085076 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:54.085076 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:54.085076 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:54.085076 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:54.086236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:54.086236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:54.086236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:54.086236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:54.086236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:54.086236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:54.086236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:54.086236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:54.086236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:54.086236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:54.086236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:54.086236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:54.086236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:54.086236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:54.086236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:54.086236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:54.086236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:54.086236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:54.087237 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:54.087237 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:54.087237 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:54.087237 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:54.087237 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:54.087237 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:54.087237 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:54.087237 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:54.087237 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:54:54.087237 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:54.091394 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 456/938] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Sep 14 01:54:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:54.092844 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec9f8d-c37b-4825-9b9f-6ba59aafd81f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:54.200680 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:54.227562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec9f8d-c37b-4825-9b9f-6ba59aafd81f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:54.265791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d821c-9c95-41a9-9b99-8523f6e7ded0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:54.267337 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d821c-9c95-41a9-9b99-8523f6e7ded0 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:54.267822 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d821c-9c95-41a9-9b99-8523f6e7ded0 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:54.268317 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d821c-9c95-41a9-9b99-8523f6e7ded0 None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:54.269246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d821c-9c95-41a9-9b99-8523f6e7ded0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:54.269772 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d821c-9c95-41a9-9b99-8523f6e7ded0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:54.271426 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d821c-9c95-41a9-9b99-8523f6e7ded0 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:54.291372 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 483/939] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Sep 14 01:54:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:54.296899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a46e57-956d-433c-a2ec-4507b64e25f8 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:54.300121 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:54.408661 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a46e57-956d-433c-a2ec-4507b64e25f8 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:54.443407 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f6a7c-e5ca-491c-84bb-bb9857d843c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:54.444136 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f6a7c-e5ca-491c-84bb-bb9857d843c5 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:54.444420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f6a7c-e5ca-491c-84bb-bb9857d843c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:54.444692 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f6a7c-e5ca-491c-84bb-bb9857d843c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:54.455630 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f6a7c-e5ca-491c-84bb-bb9857d843c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:54.457019 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f6a7c-e5ca-491c-84bb-bb9857d843c5 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:54.472131 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-cd7f6a7c-e5ca-491c-84bb-bb9857d843c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:54.472131 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:54.472131 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:54.472131 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:54.472131 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:54.472131 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:54.472131 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:54.472131 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:54.472131 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:54.472131 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:54.472131 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:54.472131 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:54.472131 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:54.472131 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:54.472131 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:54.472131 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:54.473175 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:54.473175 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:54.473175 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:54.473175 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:54.473175 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:54.473175 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:54.473175 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:54.473175 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:54.473175 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:54.473175 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:54.473175 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:54.473175 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:54.473175 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:54.473175 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:54.473175 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:54.473175 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:54.473175 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:54.474121 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:54.474121 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:54.474121 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:54.477942 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 457/940] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:54.479392 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d821c-9c95-41a9-9b99-8523f6e7ded0 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:54.586196 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d821c-9c95-41a9-9b99-8523f6e7ded0 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:54.622566 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39272356-0375-42dd-b0dc-e40e906a4696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:54.623511 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39272356-0375-42dd-b0dc-e40e906a4696 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:54.623976 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39272356-0375-42dd-b0dc-e40e906a4696 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:54.624430 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39272356-0375-42dd-b0dc-e40e906a4696 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:54.625293 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39272356-0375-42dd-b0dc-e40e906a4696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:54.626343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39272356-0375-42dd-b0dc-e40e906a4696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:54.627908 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39272356-0375-42dd-b0dc-e40e906a4696 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:54.634935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:54.646185 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 484/941] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:54.647841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f6a7c-e5ca-491c-84bb-bb9857d843c5 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:54.707137 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:54.757746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f6a7c-e5ca-491c-84bb-bb9857d843c5 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:54.794287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f491eb-a483-44d8-912d-454bf133c0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:54.794917 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f491eb-a483-44d8-912d-454bf133c0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:54.795198 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f491eb-a483-44d8-912d-454bf133c0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:54.795469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f491eb-a483-44d8-912d-454bf133c0f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:54.802806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f491eb-a483-44d8-912d-454bf133c0f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:54.804260 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f491eb-a483-44d8-912d-454bf133c0f9 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:54.815641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-f7f491eb-a483-44d8-912d-454bf133c0f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:54.815641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:54.815641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:54.815641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:54.815641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:54.815641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:54.815641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:54.815641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:54.815641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:54.815641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:54.815641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:54.815641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:54.815641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:54.815641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:54.815641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:54.816543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:54.816543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:54.816543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:54.816543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:54.816543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:54.816543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:54.816543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:54.816543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:54.816543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:54.816543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:54.816543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:54.816543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:54.816543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:54.816543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:54.816543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:54.816543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:54.816543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:54.817464 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:54.817464 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:54.817464 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:54.817464 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:54.817464 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:54.817464 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:54.817464 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:54.817464 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:54.817464 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 458/942] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:54.822549 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39272356-0375-42dd-b0dc-e40e906a4696 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:54.869035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf553c0-362c-4668-9d84-521f65d34477 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:54.869681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf553c0-362c-4668-9d84-521f65d34477 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:54.870200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf553c0-362c-4668-9d84-521f65d34477 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:54.872008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 459/943] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:54:54.877292 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629b17f-56b1-4652-b3c6-6f6c57d61251 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:54.877821 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629b17f-56b1-4652-b3c6-6f6c57d61251 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:54.878326 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629b17f-56b1-4652-b3c6-6f6c57d61251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:54.898128 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-9629b17f-56b1-4652-b3c6-6f6c57d61251 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:54.899640 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-9629b17f-56b1-4652-b3c6-6f6c57d61251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:54.932487 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39272356-0375-42dd-b0dc-e40e906a4696 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:54.977728 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 460/944] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:54:54.978529 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceeca73-c7c0-4c3a-80c1-463ce2c6380e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:54.979207 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceeca73-c7c0-4c3a-80c1-463ce2c6380e None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:54.979492 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceeca73-c7c0-4c3a-80c1-463ce2c6380e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:54.979762 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceeca73-c7c0-4c3a-80c1-463ce2c6380e None admin] PATH_INFO: `/v3/projects` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:54.980451 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceeca73-c7c0-4c3a-80c1-463ce2c6380e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:54.980755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceeca73-c7c0-4c3a-80c1-463ce2c6380e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:54.990277 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceeca73-c7c0-4c3a-80c1-463ce2c6380e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:54.996131 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 485/945] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:54:55.001669 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629b17f-56b1-4652-b3c6-6f6c57d61251 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:55.062450 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:55.108982 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629b17f-56b1-4652-b3c6-6f6c57d61251 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:55.144732 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4c522-4e2e-4cda-aafa-9dbc41089873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:55.145391 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4c522-4e2e-4cda-aafa-9dbc41089873 None admin] REQUEST_METHOD: `PUT` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:55.145672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4c522-4e2e-4cda-aafa-9dbc41089873 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:55.145945 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4c522-4e2e-4cda-aafa-9dbc41089873 None admin] PATH_INFO: `/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/e1c8d2eec886482eb135e19ded2c61ae/roles/4f2c7f12acdc45b48a2271b06f69a600` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:55.148307 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad4c522-4e2e-4cda-aafa-9dbc41089873 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e04b16dba1c4de79dc6b1a5e16ed907, user_id=e1c8d2eec886482eb135e19ded2c61ae, role_id=4f2c7f12acdc45b48a2271b06f69a600)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:55.149752 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad4c522-4e2e-4cda-aafa-9dbc41089873 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:55.164517 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 461/946] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:54:55 2022] PUT /identity/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/e1c8d2eec886482eb135e19ded2c61ae/roles/4f2c7f12acdc45b48a2271b06f69a600 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:54:55.262028 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceeca73-c7c0-4c3a-80c1-463ce2c6380e None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:55.330036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:55.373837 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceeca73-c7c0-4c3a-80c1-463ce2c6380e None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:55.417162 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9c675-6556-432f-b3db-fe3cddfb4f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:55.418182 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9c675-6556-432f-b3db-fe3cddfb4f22 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:55.418685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9c675-6556-432f-b3db-fe3cddfb4f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:55.419179 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9c675-6556-432f-b3db-fe3cddfb4f22 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:55.419964 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9c675-6556-432f-b3db-fe3cddfb4f22 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:55.421669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9c675-6556-432f-b3db-fe3cddfb4f22 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:55.431716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-b2d9c675-6556-432f-b3db-fe3cddfb4f22 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:54:55.431716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:55.431716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:54:55.431716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:54:55.431716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:54:55.431716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:54:55.431716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:54:55.431716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:55.431716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:54:55.431716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:55.431716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:54:55.431716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:55.431716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:55.431716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:55.431716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:55.431716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:55.431716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:55.432743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:55.432743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:55.432743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:55.432743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:55.432743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:55.432743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:55.432743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:54:55.432743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:54:55.432743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:54:55.432743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:54:55.432743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:55.432743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:55.432743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:55.432743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:55.432743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:55.432743 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:55.433616 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:55.433616 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:55.433616 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:55.433616 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:55.433616 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:55.433616 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:55.433616 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:55.433616 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:55.433616 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:55.433616 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:55.433616 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:54:55.433616 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:55.433616 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:54:55.433616 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:55.436417 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 486/947] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 14 01:54:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:55.442113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4c522-4e2e-4cda-aafa-9dbc41089873 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:55.548979 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4c522-4e2e-4cda-aafa-9dbc41089873 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:55.551741 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:55.590135 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37fa3da-52d8-4a70-9f7d-acec02b4a8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:55.590824 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fa3da-52d8-4a70-9f7d-acec02b4a8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:55.591102 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fa3da-52d8-4a70-9f7d-acec02b4a8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:55.591424 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fa3da-52d8-4a70-9f7d-acec02b4a8b4 None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:55.592092 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37fa3da-52d8-4a70-9f7d-acec02b4a8b4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:55.592398 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37fa3da-52d8-4a70-9f7d-acec02b4a8b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:55.593913 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37fa3da-52d8-4a70-9f7d-acec02b4a8b4 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:55.600576 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 462/948] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Sep 14 01:54:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:55.606448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9c675-6556-432f-b3db-fe3cddfb4f22 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:55.693694 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739d201-43a8-49e4-b1cc-c14fd8d60994 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:55.694039 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739d201-43a8-49e4-b1cc-c14fd8d60994 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:55.694304 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739d201-43a8-49e4-b1cc-c14fd8d60994 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:55.695905 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 463/949] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:55.700891 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f8a1f-8bf7-4a76-9224-06cbc12c5853 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:55.701174 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f8a1f-8bf7-4a76-9224-06cbc12c5853 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:55.701452 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f8a1f-8bf7-4a76-9224-06cbc12c5853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:55.714999 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9c675-6556-432f-b3db-fe3cddfb4f22 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:55.720876 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-6f8f8a1f-8bf7-4a76-9224-06cbc12c5853 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:55.722126 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-6f8f8a1f-8bf7-4a76-9224-06cbc12c5853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:55.740888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:55.758442 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad99331-e679-464a-9345-fbb46fe470f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:55.759324 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad99331-e679-464a-9345-fbb46fe470f8 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:55.759789 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad99331-e679-464a-9345-fbb46fe470f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:55.760274 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad99331-e679-464a-9345-fbb46fe470f8 None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:55.762172 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad99331-e679-464a-9345-fbb46fe470f8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:55.762508 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad99331-e679-464a-9345-fbb46fe470f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:55.763888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad99331-e679-464a-9345-fbb46fe470f8 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:55.771224 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 487/950] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Sep 14 01:54:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:55.777308 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad99331-e679-464a-9345-fbb46fe470f8 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:55.814212 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 464/951] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:54:55.889204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad99331-e679-464a-9345-fbb46fe470f8 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:55.925072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55e1fb-0fee-469f-8880-fb9faa5fc0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:55.926067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55e1fb-0fee-469f-8880-fb9faa5fc0bc None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:55.926591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55e1fb-0fee-469f-8880-fb9faa5fc0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:55.927061 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55e1fb-0fee-469f-8880-fb9faa5fc0bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:55.927962 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55e1fb-0fee-469f-8880-fb9faa5fc0bc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:55.929532 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55e1fb-0fee-469f-8880-fb9faa5fc0bc None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:55.951801 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 488/952] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 14 01:54:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:54:56.081542 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f8a1f-8bf7-4a76-9224-06cbc12c5853 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:56.191341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:56.196203 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f8a1f-8bf7-4a76-9224-06cbc12c5853 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:56.237882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55327538-fa6e-4086-9f37-e7a8497c3be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:56.238866 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55327538-fa6e-4086-9f37-e7a8497c3be4 None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:56.239369 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55327538-fa6e-4086-9f37-e7a8497c3be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:56.239838 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55327538-fa6e-4086-9f37-e7a8497c3be4 None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:56.240712 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55327538-fa6e-4086-9f37-e7a8497c3be4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:56.242360 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55327538-fa6e-4086-9f37-e7a8497c3be4 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:56.245171 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: WARNING keystone.server.flask.common [None req-55327538-fa6e-4086-9f37-e7a8497c3be4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 14 01:54:56.263358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.sql.core [None req-55327538-fa6e-4086-9f37-e7a8497c3be4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 14 01:54:56.263358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 14 01:54:56.263358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [parameters: {'id': '8639ccfcd39746139e3df07f48b21226', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 14 01:54:56.263358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81383) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 14 01:54:56.272423 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-55327538-fa6e-4086-9f37-e7a8497c3be4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8639ccfcd39746139e3df07f48b21226.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8639ccfcd39746139e3df07f48b21226. Sep 14 01:54:56.272423 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:56.272423 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 01:54:56.272423 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 01:54:56.272423 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 01:54:56.272423 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 01:54:56.272423 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 01:54:56.272423 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 01:54:56.272423 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 01:54:56.272423 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application conn.query(q) Sep 14 01:54:56.272423 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 01:54:56.272423 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 01:54:56.272423 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 01:54:56.273267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application result.read() Sep 14 01:54:56.273267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 01:54:56.273267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 01:54:56.273267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 01:54:56.273267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 01:54:56.273267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 01:54:56.273267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 01:54:56.273267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 01:54:56.273267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 01:54:56.273267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 14 01:54:56.273267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:56.273267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 14 01:54:56.273267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:56.273267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:56.273267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 14 01:54:56.273267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 14 01:54:56.273267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 14 01:54:56.273267 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 14 01:54:56.274312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 01:54:56.274312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application next(self.gen) Sep 14 01:54:56.274312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 14 01:54:56.274312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application yield resource Sep 14 01:54:56.274312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 01:54:56.274312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application next(self.gen) Sep 14 01:54:56.274312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 14 01:54:56.274312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self.session.rollback() Sep 14 01:54:56.274312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 14 01:54:56.274312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self.force_reraise() Sep 14 01:54:56.274312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 14 01:54:56.274312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise self.value Sep 14 01:54:56.274312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 14 01:54:56.274312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 14 01:54:56.274312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 14 01:54:56.274312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application session.commit() Sep 14 01:54:56.274312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 14 01:54:56.275239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 14 01:54:56.275239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 14 01:54:56.275239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._prepare_impl() Sep 14 01:54:56.275239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 14 01:54:56.275239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self.session.flush() Sep 14 01:54:56.275239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 14 01:54:56.275239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._flush(objects) Sep 14 01:54:56.275239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 14 01:54:56.275239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 14 01:54:56.275239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 14 01:54:56.275239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application compat.raise_( Sep 14 01:54:56.275239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 01:54:56.275239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception Sep 14 01:54:56.275239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 14 01:54:56.275239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application flush_context.execute() Sep 14 01:54:56.275239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 14 01:54:56.275239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rec.execute(self) Sep 14 01:54:56.276200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 14 01:54:56.276200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 14 01:54:56.276200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 14 01:54:56.276200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 14 01:54:56.276200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 14 01:54:56.276200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 14 01:54:56.276200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 14 01:54:56.276200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 14 01:54:56.276200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 14 01:54:56.276200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 14 01:54:56.276200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 14 01:54:56.276200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 14 01:54:56.276200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 14 01:54:56.276200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 14 01:54:56.276200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 14 01:54:56.277049 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 14 01:54:56.277049 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 01:54:56.277049 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception Sep 14 01:54:56.277049 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 01:54:56.277049 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 01:54:56.277049 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 01:54:56.277049 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 01:54:56.277049 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 01:54:56.277049 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 01:54:56.277049 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 01:54:56.277049 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application conn.query(q) Sep 14 01:54:56.277049 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 01:54:56.277049 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 01:54:56.277049 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 01:54:56.277049 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application result.read() Sep 14 01:54:56.277049 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 01:54:56.277049 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 01:54:56.277971 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 01:54:56.277971 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 01:54:56.277971 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 01:54:56.277971 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 01:54:56.277971 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 01:54:56.277971 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 01:54:56.277971 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 14 01:54:56.277971 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 14 01:54:56.277971 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [parameters: {'id': '8639ccfcd39746139e3df07f48b21226', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 14 01:54:56.277971 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 14 01:54:56.277971 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:56.277971 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:54:56.277971 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:56.277971 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:56.277971 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 14 01:54:56.280944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 14 01:54:56.280944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 14 01:54:56.280944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 14 01:54:56.280944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 14 01:54:56.280944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:56.280944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:54:56.280944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:56.280944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:56.280944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:56.280944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:56.280944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:56.280944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:56.280944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:56.280944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:56.280944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:56.280944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:56.280944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:56.283022 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:56.283022 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 14 01:54:56.283022 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 14 01:54:56.283022 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:56.283022 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:56.283022 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 14 01:54:56.283022 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 14 01:54:56.283022 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8639ccfcd39746139e3df07f48b21226. Sep 14 01:54:56.283022 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:56.285814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 465/953] 10.222.0.15 () {60 vars in 1140 bytes} [Wed Sep 14 01:54:56 2022] POST /identity/v3/projects => generated 262 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:56.292067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55e1fb-0fee-469f-8880-fb9faa5fc0bc None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:56.344863 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:56.411353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55e1fb-0fee-469f-8880-fb9faa5fc0bc None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:56.435200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6e5f4-376f-4e31-9093-1428e56ee23c None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:56.435846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6e5f4-376f-4e31-9093-1428e56ee23c None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:56.436218 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6e5f4-376f-4e31-9093-1428e56ee23c None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:56.441538 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 466/954] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:54:56.447111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26172ef3-0ed1-473c-af79-0f25c043ba50 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:56.447388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26172ef3-0ed1-473c-af79-0f25c043ba50 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:56.447657 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26172ef3-0ed1-473c-af79-0f25c043ba50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:56.451275 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66629508-f3d9-49a8-a0da-eea420f24c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:56.452116 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66629508-f3d9-49a8-a0da-eea420f24c4b None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:56.452573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66629508-f3d9-49a8-a0da-eea420f24c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:56.452987 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66629508-f3d9-49a8-a0da-eea420f24c4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:56.460297 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66629508-f3d9-49a8-a0da-eea420f24c4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:56.461948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66629508-f3d9-49a8-a0da-eea420f24c4b None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:56.467477 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-26172ef3-0ed1-473c-af79-0f25c043ba50 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:56.469620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-26172ef3-0ed1-473c-af79-0f25c043ba50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:56.474343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-66629508-f3d9-49a8-a0da-eea420f24c4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:56.474343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:56.474343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:56.474343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:56.474343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:56.474343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:56.474343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:56.474343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:56.474343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:56.474343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:56.474343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:56.474343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:56.474343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:56.474343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:56.474343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:56.475499 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:56.475499 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:56.475499 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:56.475499 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:56.475499 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:56.475499 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:56.475499 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:56.475499 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:56.475499 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:56.475499 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:56.475499 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:56.475499 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:56.475499 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:56.475499 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:56.475499 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:56.475499 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:56.475499 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:56.476671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:56.476671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:56.476671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:56.476671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:56.476671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:56.476671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:56.476671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:56.476671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:56.478751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 489/955] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:56.484390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66629508-f3d9-49a8-a0da-eea420f24c4b None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:56.554082 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 467/956] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:54:56.601073 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66629508-f3d9-49a8-a0da-eea420f24c4b None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:56.602492 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:56.635977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486e66e-7b2c-4031-b0c3-711bd3fedb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:56.636575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486e66e-7b2c-4031-b0c3-711bd3fedb79 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:56.636847 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486e66e-7b2c-4031-b0c3-711bd3fedb79 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:56.637118 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486e66e-7b2c-4031-b0c3-711bd3fedb79 None admin] PATH_INFO: `/v3/projects` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:56.637767 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486e66e-7b2c-4031-b0c3-711bd3fedb79 None admin] RBAC: Adding query filter params (name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:56.638096 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486e66e-7b2c-4031-b0c3-711bd3fedb79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:56.639579 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486e66e-7b2c-4031-b0c3-711bd3fedb79 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:56.656217 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 490/957] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 14 01:54:56 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:56.837537 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26172ef3-0ed1-473c-af79-0f25c043ba50 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:56.926088 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:56.948762 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26172ef3-0ed1-473c-af79-0f25c043ba50 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:56.999885 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21763b68-c4d1-448a-a8d8-e3e91b7c3b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:57.001163 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21763b68-c4d1-448a-a8d8-e3e91b7c3b4b None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:57.001439 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21763b68-c4d1-448a-a8d8-e3e91b7c3b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:57.001708 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21763b68-c4d1-448a-a8d8-e3e91b7c3b4b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:57.002318 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21763b68-c4d1-448a-a8d8-e3e91b7c3b4b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:57.003805 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21763b68-c4d1-448a-a8d8-e3e91b7c3b4b None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:57.013377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-21763b68-c4d1-448a-a8d8-e3e91b7c3b4b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 01:54:57.013377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:57.013377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:54:57.013377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:54:57.013377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:54:57.013377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:54:57.013377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:54:57.013377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:57.013377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 01:54:57.013377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:57.013377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:54:57.013377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:57.013377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:57.013377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:57.013377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:57.013377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:57.013377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:57.014555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:57.014555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:57.014555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:57.014555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:57.014555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:57.014555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:57.014555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:54:57.014555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:54:57.014555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:54:57.014555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:54:57.014555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:57.014555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:57.014555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:57.014555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:57.014555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:57.014555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:57.015465 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:57.015465 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:57.015465 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:57.015465 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:57.015465 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:57.015465 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:57.015465 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:57.015465 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:57.015465 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:57.015465 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:57.015465 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:54:57.015465 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:57.015465 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 01:54:57.015465 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:57.015465 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 468/958] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:57.020677 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486e66e-7b2c-4031-b0c3-711bd3fedb79 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:57.124194 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486e66e-7b2c-4031-b0c3-711bd3fedb79 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:57.160113 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8634b5-199c-421a-81bd-14cda5741b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:57.160727 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8634b5-199c-421a-81bd-14cda5741b4e None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:57.161008 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8634b5-199c-421a-81bd-14cda5741b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:57.161283 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8634b5-199c-421a-81bd-14cda5741b4e None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:57.161910 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8634b5-199c-421a-81bd-14cda5741b4e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:57.162244 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8634b5-199c-421a-81bd-14cda5741b4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:57.163617 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8634b5-199c-421a-81bd-14cda5741b4e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:57.173751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 491/959] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 14 01:54:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:57.179121 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21763b68-c4d1-448a-a8d8-e3e91b7c3b4b None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:57.283304 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21763b68-c4d1-448a-a8d8-e3e91b7c3b4b None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:57.323863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770fb0b-5e75-460a-b1e2-fbdc18490a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:57.324476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770fb0b-5e75-460a-b1e2-fbdc18490a6c None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:57.324782 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770fb0b-5e75-460a-b1e2-fbdc18490a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:57.325076 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770fb0b-5e75-460a-b1e2-fbdc18490a6c None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:57.325715 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770fb0b-5e75-460a-b1e2-fbdc18490a6c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:57.326023 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770fb0b-5e75-460a-b1e2-fbdc18490a6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:57.327386 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770fb0b-5e75-460a-b1e2-fbdc18490a6c None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:57.333863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 469/960] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 14 01:54:57 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:57.502879 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8cf6e-3a90-4966-98b3-defb689c9ced None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:57.503180 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8cf6e-3a90-4966-98b3-defb689c9ced None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:57.503350 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8cf6e-3a90-4966-98b3-defb689c9ced None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:57.504843 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 492/961] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:57.510499 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bd9d4-2f12-4354-86a4-de97f9285a78 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:57.510822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bd9d4-2f12-4354-86a4-de97f9285a78 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:57.511130 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bd9d4-2f12-4354-86a4-de97f9285a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:57.531031 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-642bd9d4-2f12-4354-86a4-de97f9285a78 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:57.532754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-642bd9d4-2f12-4354-86a4-de97f9285a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:57.534868 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:57.604807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 470/962] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:54:57.875776 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8cf6e-3a90-4966-98b3-defb689c9ced None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:57.904231 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:57.981239 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8cf6e-3a90-4966-98b3-defb689c9ced None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:58.017003 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b813596-7463-464a-9645-4526d0a772f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:58.017640 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b813596-7463-464a-9645-4526d0a772f5 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:58.017919 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b813596-7463-464a-9645-4526d0a772f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:58.018222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b813596-7463-464a-9645-4526d0a772f5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:58.028988 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b813596-7463-464a-9645-4526d0a772f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:54:58.034379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-3b813596-7463-464a-9645-4526d0a772f5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:54:58.034379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:58.034379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:58.034379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:58.034379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:58.034379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:58.034379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:58.034379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:58.034379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:58.034379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:58.034379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:58.034379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:58.034379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:54:58.034379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:54:58.034379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:54:58.034379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:54:58.035331 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:54:58.035331 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:54:58.035331 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:54:58.035331 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:54:58.035331 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:58.035331 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:58.035331 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:58.035331 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:58.035331 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:58.035331 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:58.035331 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:58.035331 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:58.035331 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:58.035331 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:58.035331 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:58.035331 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:58.036192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:58.036192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:58.036192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:58.036192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:58.036192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:54:58.036192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:54:58.036192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:54:58.036192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:54:58.036192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:54:58.036192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:54:58.036192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:54:58.036192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:58.036192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 493/963] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Sep 14 01:54:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:58.041322 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bd9d4-2f12-4354-86a4-de97f9285a78 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:58.144284 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bd9d4-2f12-4354-86a4-de97f9285a78 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:58.183031 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f51a6-d912-4b7f-8a44-9c2e8e6c5038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:58.183799 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f51a6-d912-4b7f-8a44-9c2e8e6c5038 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:58.183990 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f51a6-d912-4b7f-8a44-9c2e8e6c5038 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:58.184144 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f51a6-d912-4b7f-8a44-9c2e8e6c5038 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:58.184803 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f51a6-d912-4b7f-8a44-9c2e8e6c5038 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:58.185151 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f51a6-d912-4b7f-8a44-9c2e8e6c5038 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:58.186551 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f51a6-d912-4b7f-8a44-9c2e8e6c5038 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:58.195635 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 471/964] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Sep 14 01:54:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:58.195969 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:58.201379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b813596-7463-464a-9645-4526d0a772f5 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:58.268593 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8606975-9197-463d-a3b0-ebadc668a016 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:58.268930 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8606975-9197-463d-a3b0-ebadc668a016 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:58.269128 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8606975-9197-463d-a3b0-ebadc668a016 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:58.270990 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 472/965] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:54:58.277075 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d15c1-8ab2-4235-8bbc-26530f9ed883 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:58.277478 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d15c1-8ab2-4235-8bbc-26530f9ed883 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:58.277653 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d15c1-8ab2-4235-8bbc-26530f9ed883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:58.295476 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:58.298727 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-ef8d15c1-8ab2-4235-8bbc-26530f9ed883 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:58.300189 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-ef8d15c1-8ab2-4235-8bbc-26530f9ed883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:58.313043 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b813596-7463-464a-9645-4526d0a772f5 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:58.349425 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e9bc5-58a0-4f17-b2ca-b3c93dec74fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:58.350135 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e9bc5-58a0-4f17-b2ca-b3c93dec74fa None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:58.350715 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e9bc5-58a0-4f17-b2ca-b3c93dec74fa None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:58.350715 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e9bc5-58a0-4f17-b2ca-b3c93dec74fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:58.361798 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e9bc5-58a0-4f17-b2ca-b3c93dec74fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:58.363359 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e9bc5-58a0-4f17-b2ca-b3c93dec74fa None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:58.373044 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 473/966] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:54:58.385481 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-2d2e9bc5-58a0-4f17-b2ca-b3c93dec74fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:58.385481 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:58.385481 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:58.385481 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:58.385481 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:58.385481 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:58.385481 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:58.385481 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:58.385481 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:58.385481 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:58.385481 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:58.385481 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:58.385481 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:58.385481 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:58.385481 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:58.385481 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:58.386544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:58.386544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:58.386544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:58.386544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:58.386544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:58.386544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:58.386544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:58.386544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:58.386544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:58.386544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:58.386544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:58.386544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:58.386544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:58.386544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:58.386544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:58.386544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:58.386544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:58.387525 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:58.387525 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:58.387525 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:58.391038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 494/967] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:58.396932 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d15c1-8ab2-4235-8bbc-26530f9ed883 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:58.495740 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d15c1-8ab2-4235-8bbc-26530f9ed883 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:58.525844 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:58.531680 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a20149-9e5f-47a9-bed2-7786ffea74f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:58.532749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a20149-9e5f-47a9-bed2-7786ffea74f5 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:58.533373 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a20149-9e5f-47a9-bed2-7786ffea74f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:58.533651 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a20149-9e5f-47a9-bed2-7786ffea74f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:58.534340 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a20149-9e5f-47a9-bed2-7786ffea74f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:58.535016 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a20149-9e5f-47a9-bed2-7786ffea74f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:58.536742 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a20149-9e5f-47a9-bed2-7786ffea74f5 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:58.549805 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 474/968] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:58.555320 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e9bc5-58a0-4f17-b2ca-b3c93dec74fa None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:58.655255 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e9bc5-58a0-4f17-b2ca-b3c93dec74fa None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:58.661531 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a20149-9e5f-47a9-bed2-7786ffea74f5 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:58.694522 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd8e1d-1ade-49a0-883f-9849aaf42d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:58.695156 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd8e1d-1ade-49a0-883f-9849aaf42d4f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:58.695437 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd8e1d-1ade-49a0-883f-9849aaf42d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:58.695710 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd8e1d-1ade-49a0-883f-9849aaf42d4f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:58.704240 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd8e1d-1ade-49a0-883f-9849aaf42d4f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:58.705698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd8e1d-1ade-49a0-883f-9849aaf42d4f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:58.707484 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:58.717926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-55fd8e1d-1ade-49a0-883f-9849aaf42d4f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:54:58.717926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:58.717926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:58.717926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:58.717926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:58.717926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:58.717926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:58.717926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:58.717926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:58.717926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:58.717926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:58.717926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:58.717926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:54:58.717926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:54:58.717926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:54:58.717926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:54:58.719029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:58.719029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:58.719029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:58.719029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:58.719029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:58.719029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:58.719029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:58.719029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:58.719029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:58.719029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:58.719029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:58.719029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:58.719029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:58.719029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:58.719029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:58.719029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:58.719029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:58.719029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:58.720009 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:58.720009 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:58.720009 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:54:58.720009 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:54:58.720009 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:54:58.720009 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:54:58.720009 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:54:58.720009 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:54:58.720009 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:54:58.720009 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:58.722752 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 495/969] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Sep 14 01:54:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:58.727892 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd8e1d-1ade-49a0-883f-9849aaf42d4f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:58.787311 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a20149-9e5f-47a9-bed2-7786ffea74f5 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:58.823680 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d099e-c30c-4970-bb18-cdaf240a8f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:58.824754 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d099e-c30c-4970-bb18-cdaf240a8f82 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:58.825253 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d099e-c30c-4970-bb18-cdaf240a8f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:58.825738 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d099e-c30c-4970-bb18-cdaf240a8f82 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:58.826550 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d099e-c30c-4970-bb18-cdaf240a8f82 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:58.828131 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d099e-c30c-4970-bb18-cdaf240a8f82 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:58.837458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-0b3d099e-c30c-4970-bb18-cdaf240a8f82 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:54:58.837458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:58.837458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:54:58.837458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:54:58.837458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:54:58.837458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:54:58.837458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:54:58.837458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:58.837458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:54:58.837458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:58.837458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:54:58.837458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:58.837458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:58.837458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:58.837458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:58.837458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:58.837458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:58.838620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:58.838620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:58.838620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:58.838620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:58.838620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:58.838620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:58.838620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:54:58.838620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:54:58.838620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:54:58.838620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:54:58.838620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:58.838620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:58.838620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:58.838620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:58.838620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:58.838620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:58.839495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:58.839495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:58.839495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:58.839495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:58.839495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:58.839495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:58.839495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:58.839495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:58.839495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:58.839495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:58.839495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:54:58.839495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:54:58.839495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:54:58.839495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:58.842070 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 475/970] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 14 01:54:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:58.848641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d099e-c30c-4970-bb18-cdaf240a8f82 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:58.869305 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd8e1d-1ade-49a0-883f-9849aaf42d4f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:58.910475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:58.921222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e50141-42e3-4fa8-b14d-498026186b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:58.921836 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e50141-42e3-4fa8-b14d-498026186b7b None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:58.924378 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e50141-42e3-4fa8-b14d-498026186b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:58.924658 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e50141-42e3-4fa8-b14d-498026186b7b None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:58.925381 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e50141-42e3-4fa8-b14d-498026186b7b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:58.925693 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e50141-42e3-4fa8-b14d-498026186b7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:58.927513 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e50141-42e3-4fa8-b14d-498026186b7b None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:58.945653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 496/971] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Sep 14 01:54:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:54:58.955071 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e50141-42e3-4fa8-b14d-498026186b7b None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:58.969824 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d099e-c30c-4970-bb18-cdaf240a8f82 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:59.010328 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc81f3-0927-463b-97d4-b473004ad9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:59.011254 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc81f3-0927-463b-97d4-b473004ad9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:59.011537 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc81f3-0927-463b-97d4-b473004ad9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:59.011814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc81f3-0927-463b-97d4-b473004ad9c4 None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:59.012442 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc81f3-0927-463b-97d4-b473004ad9c4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:59.012747 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc81f3-0927-463b-97d4-b473004ad9c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:59.013336 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:59.014133 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc81f3-0927-463b-97d4-b473004ad9c4 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:59.020348 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 476/972] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Sep 14 01:54:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:59.021640 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c472e8c-d49a-4339-a47d-64898fb9a713 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:59.021932 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c472e8c-d49a-4339-a47d-64898fb9a713 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:59.022315 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c472e8c-d49a-4339-a47d-64898fb9a713 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:59.026016 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 477/973] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:59.028060 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c472e8c-d49a-4339-a47d-64898fb9a713 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:59.066624 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e50141-42e3-4fa8-b14d-498026186b7b None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:59.101352 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444d518-fca4-4506-b3a5-564106ee9b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:59.102004 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444d518-fca4-4506-b3a5-564106ee9b20 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:59.102283 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444d518-fca4-4506-b3a5-564106ee9b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:59.102553 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444d518-fca4-4506-b3a5-564106ee9b20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:59.134983 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c472e8c-d49a-4339-a47d-64898fb9a713 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:59.152257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:59.172854 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952a5f3-6d10-432e-aeb5-03e7aa1d118d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:59.173372 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952a5f3-6d10-432e-aeb5-03e7aa1d118d None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:59.173645 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952a5f3-6d10-432e-aeb5-03e7aa1d118d None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:59.173911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952a5f3-6d10-432e-aeb5-03e7aa1d118d None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:59.174616 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952a5f3-6d10-432e-aeb5-03e7aa1d118d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:59.174915 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952a5f3-6d10-432e-aeb5-03e7aa1d118d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:59.175028 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444d518-fca4-4506-b3a5-564106ee9b20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:59.176264 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952a5f3-6d10-432e-aeb5-03e7aa1d118d None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:59.176457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444d518-fca4-4506-b3a5-564106ee9b20 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:59.182118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 478/974] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Sep 14 01:54:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:59.183052 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d38d0-8da8-4b8d-81aa-3f474bb7861b None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:59.183332 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d38d0-8da8-4b8d-81aa-3f474bb7861b None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:59.183856 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d38d0-8da8-4b8d-81aa-3f474bb7861b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:59.201107 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-690d38d0-8da8-4b8d-81aa-3f474bb7861b None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:59.202319 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-690d38d0-8da8-4b8d-81aa-3f474bb7861b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:59.243975 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:59.272645 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 479/975] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:54:59.273482 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d38d0-8da8-4b8d-81aa-3f474bb7861b None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:59.306135 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 497/976] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:54:58 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:54:59.310234 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444d518-fca4-4506-b3a5-564106ee9b20 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:59.413459 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d38d0-8da8-4b8d-81aa-3f474bb7861b None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:59.417100 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444d518-fca4-4506-b3a5-564106ee9b20 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:59.431011 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:59.454538 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8386779-3070-49be-9844-047bd2701272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:59.455482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8386779-3070-49be-9844-047bd2701272 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:59.455964 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8386779-3070-49be-9844-047bd2701272 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:59.456430 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8386779-3070-49be-9844-047bd2701272 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:59.459885 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f83b0-d32b-4e06-89b4-3e5258e04e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:59.460486 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f83b0-d32b-4e06-89b4-3e5258e04e80 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:59.460763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f83b0-d32b-4e06-89b4-3e5258e04e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:59.461033 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f83b0-d32b-4e06-89b4-3e5258e04e80 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:59.461738 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f83b0-d32b-4e06-89b4-3e5258e04e80 None admin] RBAC: Adding query filter params (interface=admin, service_id=67b6d7123d314ff39290b96a553be85e, region_id=RegionOne) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:59.462062 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f83b0-d32b-4e06-89b4-3e5258e04e80 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:59.463426 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f83b0-d32b-4e06-89b4-3e5258e04e80 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:59.469307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8386779-3070-49be-9844-047bd2701272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:59.471051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8386779-3070-49be-9844-047bd2701272 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:59.474195 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 480/977] 10.222.0.15 () {56 vars in 1249 bytes} [Wed Sep 14 01:54:59 2022] GET /identity/v3/endpoints?service_id=67b6d7123d314ff39290b96a553be85e&interface=admin®ion_id=RegionOne => generated 193 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:54:59.475989 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f83b0-d32b-4e06-89b4-3e5258e04e80 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:59.495653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-f8386779-3070-49be-9844-047bd2701272 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:59.495653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:59.495653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:59.495653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:59.495653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:59.495653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:59.495653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:59.495653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:59.495653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:59.495653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:59.495653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:59.495653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:59.495653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:54:59.495653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:54:59.495653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:54:59.495653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:54:59.496686 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:59.496686 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:59.496686 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:59.496686 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:59.496686 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:59.496686 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:59.496686 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:59.496686 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:59.496686 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:59.496686 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:59.496686 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:59.496686 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:59.496686 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:59.496686 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:59.496686 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:59.496686 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:59.496686 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:54:59.497714 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:54:59.497714 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:54:59.497714 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:59.505392 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 498/978] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:54:59.506550 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8386779-3070-49be-9844-047bd2701272 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:59.521711 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:59.609246 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f83b0-d32b-4e06-89b4-3e5258e04e80 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:59.619230 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8386779-3070-49be-9844-047bd2701272 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:59.645080 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f61075-2e4b-454b-9545-3702ee3006e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:59.645681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f61075-2e4b-454b-9545-3702ee3006e7 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:59.645952 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f61075-2e4b-454b-9545-3702ee3006e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:59.646258 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f61075-2e4b-454b-9545-3702ee3006e7 None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:59.646855 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f61075-2e4b-454b-9545-3702ee3006e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:59.648208 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f61075-2e4b-454b-9545-3702ee3006e7 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:59.653674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c678f0-747b-4a3d-8868-7a416e07a142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:59.666013 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 481/979] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Sep 14 01:54:59 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:54:59.666245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c678f0-747b-4a3d-8868-7a416e07a142 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:59.666245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c678f0-747b-4a3d-8868-7a416e07a142 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:59.666419 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f61075-2e4b-454b-9545-3702ee3006e7 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:59.666535 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c678f0-747b-4a3d-8868-7a416e07a142 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:59.666535 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c678f0-747b-4a3d-8868-7a416e07a142 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:54:59.666535 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c678f0-747b-4a3d-8868-7a416e07a142 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:59.666535 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c678f0-747b-4a3d-8868-7a416e07a142 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:59.670718 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 499/980] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:54:59.672737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e9352-41b7-46c1-a766-68935044b79e None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:59.673020 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e9352-41b7-46c1-a766-68935044b79e None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:59.673286 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e9352-41b7-46c1-a766-68935044b79e None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:59.674503 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 500/981] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:54:59.679504 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e9352-41b7-46c1-a766-68935044b79e None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:59.711894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:59.756364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:54:59.780913 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f61075-2e4b-454b-9545-3702ee3006e7 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:59.797114 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e9352-41b7-46c1-a766-68935044b79e None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:54:59.822929 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93494c4e-c5de-4f37-b612-e60f0874c74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:59.823561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93494c4e-c5de-4f37-b612-e60f0874c74d None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:59.823827 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93494c4e-c5de-4f37-b612-e60f0874c74d None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:59.824113 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93494c4e-c5de-4f37-b612-e60f0874c74d None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:59.824741 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93494c4e-c5de-4f37-b612-e60f0874c74d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:59.826224 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93494c4e-c5de-4f37-b612-e60f0874c74d None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:59.838505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3eb4a0-2ca5-43c5-b253-5e5565b547ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:54:59.839149 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3eb4a0-2ca5-43c5-b253-5e5565b547ed None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:59.839448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3eb4a0-2ca5-43c5-b253-5e5565b547ed None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:59.839690 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3eb4a0-2ca5-43c5-b253-5e5565b547ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:59.841943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.sql.core [None req-93494c4e-c5de-4f37-b612-e60f0874c74d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 14 01:54:59.841943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 14 01:54:59.841943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [parameters: {'user_id': '5dd83a510efd4d15bb583450db76dafa', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 14 01:54:59.841943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81383) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 14 01:54:59.848583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-93494c4e-c5de-4f37-b612-e60f0874c74d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 14 01:54:59.848583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:59.848583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 01:54:59.848583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 01:54:59.848583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 01:54:59.848583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 01:54:59.848583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 01:54:59.848583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 01:54:59.848583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 01:54:59.848583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application conn.query(q) Sep 14 01:54:59.848583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 01:54:59.848583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 01:54:59.848583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 01:54:59.848583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application result.read() Sep 14 01:54:59.848583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 01:54:59.849472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 01:54:59.849472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 01:54:59.849472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 01:54:59.849472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 01:54:59.849472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 01:54:59.849472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 01:54:59.849472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 01:54:59.849472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 14 01:54:59.849472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:59.849472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 14 01:54:59.849472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:59.849472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:59.849472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 14 01:54:59.849472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 14 01:54:59.849472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 14 01:54:59.849472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 14 01:54:59.849472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 01:54:59.849472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application next(self.gen) Sep 14 01:54:59.851244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 14 01:54:59.851244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application yield resource Sep 14 01:54:59.851244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 01:54:59.851244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application next(self.gen) Sep 14 01:54:59.851244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 14 01:54:59.851244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self.session.rollback() Sep 14 01:54:59.851244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 14 01:54:59.851244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self.force_reraise() Sep 14 01:54:59.851244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 14 01:54:59.851244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise self.value Sep 14 01:54:59.851244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 14 01:54:59.851244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 14 01:54:59.851244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 14 01:54:59.851244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application session.commit() Sep 14 01:54:59.851244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 14 01:54:59.851244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 14 01:54:59.851244 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 14 01:54:59.852202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._prepare_impl() Sep 14 01:54:59.852202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 14 01:54:59.852202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self.session.flush() Sep 14 01:54:59.852202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 14 01:54:59.852202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._flush(objects) Sep 14 01:54:59.852202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 14 01:54:59.852202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 14 01:54:59.852202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 14 01:54:59.852202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application compat.raise_( Sep 14 01:54:59.852202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 01:54:59.852202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception Sep 14 01:54:59.852202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 14 01:54:59.852202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application flush_context.execute() Sep 14 01:54:59.852202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 14 01:54:59.852202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rec.execute(self) Sep 14 01:54:59.852202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 14 01:54:59.852202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 14 01:54:59.853209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 14 01:54:59.853209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 14 01:54:59.853209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 14 01:54:59.853209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 14 01:54:59.853209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 14 01:54:59.853209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 14 01:54:59.853209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 14 01:54:59.853209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 14 01:54:59.853209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 14 01:54:59.853209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 14 01:54:59.853209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 14 01:54:59.853209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 14 01:54:59.853209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 14 01:54:59.853209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 14 01:54:59.853209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 01:54:59.853209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception Sep 14 01:54:59.854187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 01:54:59.854187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 01:54:59.854187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 01:54:59.854187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 01:54:59.854187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 01:54:59.854187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 01:54:59.854187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 01:54:59.854187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application conn.query(q) Sep 14 01:54:59.854187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 01:54:59.854187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 01:54:59.854187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 01:54:59.854187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application result.read() Sep 14 01:54:59.854187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 01:54:59.854187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 01:54:59.854187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 01:54:59.854187 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 01:54:59.856404 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 01:54:59.856404 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 01:54:59.856404 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 01:54:59.856404 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 01:54:59.856404 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 14 01:54:59.856404 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 14 01:54:59.856404 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [parameters: {'user_id': '5dd83a510efd4d15bb583450db76dafa', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 14 01:54:59.856404 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 14 01:54:59.856404 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:59.856404 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:54:59.856404 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:59.856404 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:59.856404 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:59.856404 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:59.856404 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:59.858959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:59.858959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:59.858959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:59.858959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:59.858959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:59.858959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:59.858959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:59.858959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 14 01:54:59.858959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 14 01:54:59.858959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:59.858959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:59.858959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:54:59.858959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:59.858959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:54:59.858959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:54:59.858959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 14 01:54:59.858959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 14 01:54:59.860779 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 14 01:54:59.860779 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 14 01:54:59.860779 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 14 01:54:59.860779 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 14 01:54:59.860779 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 14 01:54:59.860779 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:54:59.861372 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3eb4a0-2ca5-43c5-b253-5e5565b547ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:54:59.861459 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 482/982] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Sep 14 01:54:59 2022] POST /identity/v3/users => generated 160 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:59.861536 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3eb4a0-2ca5-43c5-b253-5e5565b547ed None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:54:59.861606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbc475-4c6b-4871-89ea-99b331de74fd None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:54:59.861606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbc475-4c6b-4871-89ea-99b331de74fd None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:54:59.861606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbc475-4c6b-4871-89ea-99b331de74fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:54:59.867684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-2f3eb4a0-2ca5-43c5-b253-5e5565b547ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:59.867684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:54:59.867684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:54:59.867684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:54:59.867684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:54:59.867684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:54:59.867684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:54:59.867684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:54:59.867684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:54:59.867684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:54:59.867684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:54:59.867684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:54:59.867684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:54:59.867684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:54:59.867684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:54:59.868746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:54:59.868746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:54:59.868746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:54:59.868746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:54:59.868746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:54:59.868746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:54:59.868746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:54:59.868746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:54:59.868746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:54:59.868746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:54:59.868746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:54:59.868746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:54:59.868746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:54:59.868746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:54:59.868746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:54:59.868746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:54:59.868746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:54:59.870534 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:54:59.870534 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:54:59.870534 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:54:59.870534 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:54:59.870534 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:54:59.870534 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:54:59.870534 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:54:59.870534 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:54:59.870534 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 501/983] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:54:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:54:59.871786 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3eb4a0-2ca5-43c5-b253-5e5565b547ed None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:59.879510 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-6adbc475-4c6b-4871-89ea-99b331de74fd None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:54:59.882513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-6adbc475-4c6b-4871-89ea-99b331de74fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:54:59.957838 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 483/984] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:54:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:54:59.958761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbc475-4c6b-4871-89ea-99b331de74fd None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:54:59.988436 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3eb4a0-2ca5-43c5-b253-5e5565b547ed None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:00.012507 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:00.021253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:00.032616 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97c0c7-b91b-4d55-8f70-fa6510d3acec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:00.033595 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97c0c7-b91b-4d55-8f70-fa6510d3acec None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:00.034077 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97c0c7-b91b-4d55-8f70-fa6510d3acec None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:00.034500 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97c0c7-b91b-4d55-8f70-fa6510d3acec None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:00.044050 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97c0c7-b91b-4d55-8f70-fa6510d3acec None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:00.045823 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97c0c7-b91b-4d55-8f70-fa6510d3acec None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:00.058763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-9f97c0c7-b91b-4d55-8f70-fa6510d3acec None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 01:55:00.058763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:00.058763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:00.058763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:00.058763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:00.058763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:00.058763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:00.058763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:00.058763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:00.058763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:00.058763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:00.058763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:00.058763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:55:00.058763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:55:00.058763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:55:00.058763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:55:00.059805 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:00.059805 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:00.059805 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:55:00.059805 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:55:00.059805 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:55:00.059805 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:55:00.059805 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:00.059805 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:00.059805 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:00.059805 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:00.059805 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:00.059805 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:00.059805 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:00.059805 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:00.059805 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:00.059805 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:00.059805 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:00.059805 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:00.060807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:00.060807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:00.060807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:55:00.060807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:55:00.060807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:55:00.060807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:55:00.060807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:55:00.060807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:55:00.060807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 01:55:00.060807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:55:00.063665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 502/985] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:54:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:55:00.069052 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97c0c7-b91b-4d55-8f70-fa6510d3acec None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:00.102345 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbc475-4c6b-4871-89ea-99b331de74fd None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:00.138935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba70d0-d67b-489d-8382-708c268d5c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:00.139657 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba70d0-d67b-489d-8382-708c268d5c95 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:00.139939 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba70d0-d67b-489d-8382-708c268d5c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:00.140495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba70d0-d67b-489d-8382-708c268d5c95 None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:00.141576 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba70d0-d67b-489d-8382-708c268d5c95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:00.141883 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba70d0-d67b-489d-8382-708c268d5c95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:00.143556 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba70d0-d67b-489d-8382-708c268d5c95 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:00.165188 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 484/986] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:54:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:00.166294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba70d0-d67b-489d-8382-708c268d5c95 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:00.193754 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97c0c7-b91b-4d55-8f70-fa6510d3acec None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:00.237666 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4a65e-f35d-4a2b-888a-669aa1918c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:00.238381 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4a65e-f35d-4a2b-888a-669aa1918c5d None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:00.238708 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4a65e-f35d-4a2b-888a-669aa1918c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:00.238939 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4a65e-f35d-4a2b-888a-669aa1918c5d None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:00.239633 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4a65e-f35d-4a2b-888a-669aa1918c5d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:00.239930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4a65e-f35d-4a2b-888a-669aa1918c5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:00.241356 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4a65e-f35d-4a2b-888a-669aa1918c5d None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:00.265291 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:00.276576 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba70d0-d67b-489d-8382-708c268d5c95 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:00.278021 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 503/987] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 14 01:55:00 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:55:00.278493 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4a65e-f35d-4a2b-888a-669aa1918c5d None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:00.330829 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:00.338763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31e8a2-4a72-49d4-8211-3b1d07b3a920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:00.340428 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31e8a2-4a72-49d4-8211-3b1d07b3a920 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:00.341213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31e8a2-4a72-49d4-8211-3b1d07b3a920 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:00.341725 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31e8a2-4a72-49d4-8211-3b1d07b3a920 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:00.343048 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31e8a2-4a72-49d4-8211-3b1d07b3a920 None admin] RBAC: Adding query filter params (role.id=4f2c7f12acdc45b48a2271b06f69a600, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=e1c8d2eec886482eb135e19ded2c61ae) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:00.344109 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31e8a2-4a72-49d4-8211-3b1d07b3a920 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:00.346517 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31e8a2-4a72-49d4-8211-3b1d07b3a920 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:00.366275 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 485/988] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:55:00 2022] GET /identity/v3/role_assignments?user.id=e1c8d2eec886482eb135e19ded2c61ae&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=4f2c7f12acdc45b48a2271b06f69a600 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:00.419097 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4a65e-f35d-4a2b-888a-669aa1918c5d None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:00.459565 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f7e67-abd7-4a73-abe4-7b1c8e8e3f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:00.460203 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f7e67-abd7-4a73-abe4-7b1c8e8e3f47 None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:00.460480 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f7e67-abd7-4a73-abe4-7b1c8e8e3f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:00.460750 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f7e67-abd7-4a73-abe4-7b1c8e8e3f47 None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:00.461420 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f7e67-abd7-4a73-abe4-7b1c8e8e3f47 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:00.462971 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f7e67-abd7-4a73-abe4-7b1c8e8e3f47 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:00.481786 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 504/989] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 14 01:55:00 2022] POST /identity/v3/services => generated 242 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:55:01.859261 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a4f37-f62c-4339-8f4c-820c33831263 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:01.859793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a4f37-f62c-4339-8f4c-820c33831263 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:01.859875 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a4f37-f62c-4339-8f4c-820c33831263 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:01.861434 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 486/990] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:55:01.866673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79eb66a-9ebb-4db0-a2a6-0578b7d19d7c None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:01.867086 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79eb66a-9ebb-4db0-a2a6-0578b7d19d7c None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:01.867252 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79eb66a-9ebb-4db0-a2a6-0578b7d19d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:01.885721 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-a79eb66a-9ebb-4db0-a2a6-0578b7d19d7c None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:01.886928 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-a79eb66a-9ebb-4db0-a2a6-0578b7d19d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:01.959498 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 505/991] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:55:02.223207 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a4f37-f62c-4339-8f4c-820c33831263 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:02.329130 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a4f37-f62c-4339-8f4c-820c33831263 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:02.362701 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8e264-e202-4bb1-9958-a08102cfa355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:02.363328 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8e264-e202-4bb1-9958-a08102cfa355 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:02.363606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8e264-e202-4bb1-9958-a08102cfa355 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:02.363876 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8e264-e202-4bb1-9958-a08102cfa355 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:02.364452 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8e264-e202-4bb1-9958-a08102cfa355 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:02.365811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8e264-e202-4bb1-9958-a08102cfa355 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:02.374534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-84d8e264-e202-4bb1-9958-a08102cfa355 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:55:02.374534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:02.374534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:55:02.374534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:55:02.374534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:55:02.374534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:55:02.374534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:55:02.374534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:55:02.374534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:55:02.374534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:02.374534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:55:02.374534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:02.374534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:02.374534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:02.374534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:02.374534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:02.374534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:02.375769 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:02.375769 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:02.375769 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:02.375769 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:02.375769 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:02.375769 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:02.375769 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:55:02.375769 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:55:02.375769 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:55:02.375769 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:55:02.375769 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:02.375769 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:02.375769 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:02.375769 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:02.375769 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:02.375769 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:02.376632 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:02.376632 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:02.376632 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:02.376632 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:02.376632 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:02.376632 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:02.376632 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:02.376632 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:02.376632 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:02.376632 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:02.376632 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:55:02.376632 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:55:02.376632 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:55:02.376632 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:02.376632 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 487/992] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 14 01:55:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:55:02.382367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79eb66a-9ebb-4db0-a2a6-0578b7d19d7c None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:02.415654 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:02.488134 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79eb66a-9ebb-4db0-a2a6-0578b7d19d7c None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:02.524619 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aac25f-9a07-4b4a-a0aa-8932041e6661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:02.525223 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aac25f-9a07-4b4a-a0aa-8932041e6661 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:02.525497 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aac25f-9a07-4b4a-a0aa-8932041e6661 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:02.525764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aac25f-9a07-4b4a-a0aa-8932041e6661 None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:02.526427 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aac25f-9a07-4b4a-a0aa-8932041e6661 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:02.526725 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aac25f-9a07-4b4a-a0aa-8932041e6661 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:02.528089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aac25f-9a07-4b4a-a0aa-8932041e6661 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:02.532982 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 506/993] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Sep 14 01:55:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:02.538444 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8e264-e202-4bb1-9958-a08102cfa355 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:02.561596 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:02.597703 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbaab28-406b-429c-904e-b4c1cbc18b98 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:02.598080 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbaab28-406b-429c-904e-b4c1cbc18b98 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:02.598282 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbaab28-406b-429c-904e-b4c1cbc18b98 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:02.599801 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 507/994] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:55:02.604413 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d4496-8cb5-4710-b716-b6de2029a8c5 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:02.604697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d4496-8cb5-4710-b716-b6de2029a8c5 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:02.604975 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d4496-8cb5-4710-b716-b6de2029a8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:02.625342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-6c1d4496-8cb5-4710-b716-b6de2029a8c5 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:02.627107 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-6c1d4496-8cb5-4710-b716-b6de2029a8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:02.645825 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8e264-e202-4bb1-9958-a08102cfa355 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:02.679846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd24dc-73d0-4e8b-8697-c29339d959ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:02.680448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd24dc-73d0-4e8b-8697-c29339d959ef None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:02.680724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd24dc-73d0-4e8b-8697-c29339d959ef None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:02.681014 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd24dc-73d0-4e8b-8697-c29339d959ef None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:02.681624 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd24dc-73d0-4e8b-8697-c29339d959ef None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:02.681918 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd24dc-73d0-4e8b-8697-c29339d959ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:02.683320 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd24dc-73d0-4e8b-8697-c29339d959ef None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:02.688567 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 488/995] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Sep 14 01:55:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:02.689687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391597b7-f5b3-43e7-8b65-5aadd6c23038 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:02.690089 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391597b7-f5b3-43e7-8b65-5aadd6c23038 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:02.690271 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391597b7-f5b3-43e7-8b65-5aadd6c23038 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:02.691704 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 489/996] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:55:02.695293 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391597b7-f5b3-43e7-8b65-5aadd6c23038 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:02.698908 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 508/997] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:55:02.700168 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f9ab1-1b6c-4b66-90a4-3f985d80e8be None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:02.700456 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f9ab1-1b6c-4b66-90a4-3f985d80e8be None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:02.700728 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f9ab1-1b6c-4b66-90a4-3f985d80e8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:02.717914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-384f9ab1-1b6c-4b66-90a4-3f985d80e8be None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:02.719645 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-384f9ab1-1b6c-4b66-90a4-3f985d80e8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:02.740615 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:02.784016 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:02.794040 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 509/998] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:55:02.803653 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391597b7-f5b3-43e7-8b65-5aadd6c23038 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:02.838431 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9439d9-31c8-4dbb-88b8-ac2b4cdd3468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:02.839035 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9439d9-31c8-4dbb-88b8-ac2b4cdd3468 None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:02.839313 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9439d9-31c8-4dbb-88b8-ac2b4cdd3468 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:02.839584 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9439d9-31c8-4dbb-88b8-ac2b4cdd3468 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:02.840238 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9439d9-31c8-4dbb-88b8-ac2b4cdd3468 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:02.841598 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9439d9-31c8-4dbb-88b8-ac2b4cdd3468 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:02.858763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 490/999] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 14 01:55:02 2022] POST /identity/v3/endpoints => generated 325 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:55:02.967595 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f9ab1-1b6c-4b66-90a4-3f985d80e8be None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:03.067369 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9439d9-31c8-4dbb-88b8-ac2b4cdd3468 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:03.074217 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f9ab1-1b6c-4b66-90a4-3f985d80e8be None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:03.122581 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf5ca1-35df-4780-9924-e65de05ba201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:03.123852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf5ca1-35df-4780-9924-e65de05ba201 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:03.124342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf5ca1-35df-4780-9924-e65de05ba201 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:03.125637 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf5ca1-35df-4780-9924-e65de05ba201 None admin] PATH_INFO: `/v3/services/image` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:03.126871 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf5ca1-35df-4780-9924-e65de05ba201 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:03.128944 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf5ca1-35df-4780-9924-e65de05ba201 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:03.139145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-56bf5ca1-35df-4780-9924-e65de05ba201 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 01:55:03.139145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:03.139145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:55:03.139145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:55:03.139145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:55:03.139145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:55:03.139145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:55:03.139145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:55:03.139145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 01:55:03.139145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:55:03.139145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:55:03.139145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:55:03.139145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:03.139145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:03.139145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:03.139145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:03.139145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:03.140312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:03.140312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:03.140312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:03.140312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:03.140312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:03.140312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:03.140312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:55:03.140312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:55:03.140312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:55:03.140312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:55:03.140312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:03.140312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:03.140312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:03.140312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:03.140312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:03.140312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:03.141326 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:03.141326 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:03.141326 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:03.141326 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:03.141326 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:03.141326 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:03.141326 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:03.141326 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:03.141326 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:03.141326 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:03.141326 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:55:03.141326 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:55:03.141326 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 01:55:03.141326 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:55:03.143814 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 510/1000] 10.222.0.15 () {56 vars in 1105 bytes} [Wed Sep 14 01:55:02 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:55:03.149370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf5ca1-35df-4780-9924-e65de05ba201 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:03.167888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:03.174130 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9439d9-31c8-4dbb-88b8-ac2b4cdd3468 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:03.210127 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67eb545-c97f-4964-abdc-fb3bc64b586d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:03.210775 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67eb545-c97f-4964-abdc-fb3bc64b586d None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:03.211046 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67eb545-c97f-4964-abdc-fb3bc64b586d None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:03.211321 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67eb545-c97f-4964-abdc-fb3bc64b586d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:03.211901 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67eb545-c97f-4964-abdc-fb3bc64b586d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:03.213262 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67eb545-c97f-4964-abdc-fb3bc64b586d None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:03.222834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-a67eb545-c97f-4964-abdc-fb3bc64b586d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 01:55:03.222834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:03.222834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:55:03.222834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:55:03.222834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:55:03.222834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:55:03.222834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:55:03.222834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:55:03.222834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 01:55:03.222834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:03.222834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:55:03.222834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:03.222834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:03.222834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:03.222834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:03.222834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:03.222834 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:03.223943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:03.223943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:03.223943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:03.223943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:03.223943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:03.223943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:03.223943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:55:03.223943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:55:03.223943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:55:03.223943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:55:03.223943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:03.223943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:03.223943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:03.223943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:03.223943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:03.223943 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:03.224815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:03.224815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:03.224815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:03.224815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:03.224815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:03.224815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:03.224815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:03.224815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:03.224815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:03.224815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:03.224815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:55:03.224815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:55:03.224815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 01:55:03.224815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:03.224815 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 491/1001] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:55:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:55:03.229882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67eb545-c97f-4964-abdc-fb3bc64b586d None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:03.243460 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:03.262078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf5ca1-35df-4780-9924-e65de05ba201 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:03.298198 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b98f3d-efff-46d3-856c-7c6f9a873819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:03.299118 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b98f3d-efff-46d3-856c-7c6f9a873819 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:03.299646 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b98f3d-efff-46d3-856c-7c6f9a873819 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:03.300124 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b98f3d-efff-46d3-856c-7c6f9a873819 None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:03.301062 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b98f3d-efff-46d3-856c-7c6f9a873819 None admin] RBAC: Adding query filter params (name=image) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:03.301571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b98f3d-efff-46d3-856c-7c6f9a873819 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:03.303265 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b98f3d-efff-46d3-856c-7c6f9a873819 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:03.309428 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 511/1002] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:55:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:55:03.315312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b98f3d-efff-46d3-856c-7c6f9a873819 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:03.344387 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67eb545-c97f-4964-abdc-fb3bc64b586d None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:03.383957 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9eeb1e-75e2-48a2-ab06-65da06be3ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:03.385275 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9eeb1e-75e2-48a2-ab06-65da06be3ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:03.385681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9eeb1e-75e2-48a2-ab06-65da06be3ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:03.385967 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9eeb1e-75e2-48a2-ab06-65da06be3ac4 None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:03.386770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9eeb1e-75e2-48a2-ab06-65da06be3ac4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:03.387077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9eeb1e-75e2-48a2-ab06-65da06be3ac4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:03.388455 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9eeb1e-75e2-48a2-ab06-65da06be3ac4 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:03.400692 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 492/1003] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 14 01:55:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:55:03.401471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9eeb1e-75e2-48a2-ab06-65da06be3ac4 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:03.409658 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:03.432556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b98f3d-efff-46d3-856c-7c6f9a873819 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:03.469097 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32cf3c-adb2-4d7f-b473-2c01a4fbc6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:03.469701 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32cf3c-adb2-4d7f-b473-2c01a4fbc6c6 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:03.469998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32cf3c-adb2-4d7f-b473-2c01a4fbc6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:03.470283 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32cf3c-adb2-4d7f-b473-2c01a4fbc6c6 None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:03.471013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32cf3c-adb2-4d7f-b473-2c01a4fbc6c6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:03.471339 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32cf3c-adb2-4d7f-b473-2c01a4fbc6c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:03.471640 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:03.473340 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32cf3c-adb2-4d7f-b473-2c01a4fbc6c6 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:03.486012 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 512/1004] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:55:03 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:55:03.513678 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9eeb1e-75e2-48a2-ab06-65da06be3ac4 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:03.551266 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1c858-6771-47fd-abcd-1e6461fd6c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:03.551896 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1c858-6771-47fd-abcd-1e6461fd6c3d None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:03.552178 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1c858-6771-47fd-abcd-1e6461fd6c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:03.552450 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1c858-6771-47fd-abcd-1e6461fd6c3d None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:03.553098 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca1c858-6771-47fd-abcd-1e6461fd6c3d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:03.553400 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca1c858-6771-47fd-abcd-1e6461fd6c3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:03.554840 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca1c858-6771-47fd-abcd-1e6461fd6c3d None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:03.566105 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 493/1005] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 14 01:55:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:03.567060 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32cf3c-adb2-4d7f-b473-2c01a4fbc6c6 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:03.675907 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32cf3c-adb2-4d7f-b473-2c01a4fbc6c6 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:03.710194 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00ba75-b031-4a9f-81d9-eb3f1d439de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:03.710807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00ba75-b031-4a9f-81d9-eb3f1d439de8 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:03.711087 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00ba75-b031-4a9f-81d9-eb3f1d439de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:03.711360 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00ba75-b031-4a9f-81d9-eb3f1d439de8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:03.712045 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac00ba75-b031-4a9f-81d9-eb3f1d439de8 None admin] RBAC: Adding query filter params (interface=public, service_id=18b55a11be594e6c82af9394b210c4de, region_id=RegionOne) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:03.712355 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac00ba75-b031-4a9f-81d9-eb3f1d439de8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:03.713724 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac00ba75-b031-4a9f-81d9-eb3f1d439de8 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:03.723404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 513/1006] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Sep 14 01:55:03 2022] GET /identity/v3/endpoints?service_id=18b55a11be594e6c82af9394b210c4de&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:55:03.724954 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1c858-6771-47fd-abcd-1e6461fd6c3d None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:03.827750 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1c858-6771-47fd-abcd-1e6461fd6c3d None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:03.862369 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243ca6a-4a52-46ed-a393-2491e003d754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:03.862974 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243ca6a-4a52-46ed-a393-2491e003d754 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:03.863251 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243ca6a-4a52-46ed-a393-2491e003d754 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:03.863521 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243ca6a-4a52-46ed-a393-2491e003d754 None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:03.864116 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243ca6a-4a52-46ed-a393-2491e003d754 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:03.865471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243ca6a-4a52-46ed-a393-2491e003d754 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:03.867583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:03.878087 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 494/1007] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Sep 14 01:55:03 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:55:05.039393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a2863-9fa8-4bab-bb7b-80b5458cdd04 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:05.039914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a2863-9fa8-4bab-bb7b-80b5458cdd04 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:05.039995 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a2863-9fa8-4bab-bb7b-80b5458cdd04 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:05.041402 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 514/1008] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:55:05.046483 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de45d5-fbf0-4861-a130-e1636f3fea46 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:05.046862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de45d5-fbf0-4861-a130-e1636f3fea46 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:05.047072 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de45d5-fbf0-4861-a130-e1636f3fea46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:05.065717 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-14de45d5-fbf0-4861-a130-e1636f3fea46 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:05.067061 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-14de45d5-fbf0-4861-a130-e1636f3fea46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:05.135845 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 495/1009] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:05 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:55:05.408384 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a2863-9fa8-4bab-bb7b-80b5458cdd04 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:05.419712 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:05.511016 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a2863-9fa8-4bab-bb7b-80b5458cdd04 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:05.547248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69eb3e7-1f7b-4f17-8201-30b9cb04d2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:05.547907 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69eb3e7-1f7b-4f17-8201-30b9cb04d2cd None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:05.548184 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69eb3e7-1f7b-4f17-8201-30b9cb04d2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:05.548453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69eb3e7-1f7b-4f17-8201-30b9cb04d2cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:05.549630 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69eb3e7-1f7b-4f17-8201-30b9cb04d2cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:05.551170 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69eb3e7-1f7b-4f17-8201-30b9cb04d2cd None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:05.552997 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 515/1010] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:55:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:05.558809 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de45d5-fbf0-4861-a130-e1636f3fea46 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:05.650500 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d0c2b9-c12f-40b4-a004-916bea47add5 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:05.650674 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d0c2b9-c12f-40b4-a004-916bea47add5 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:05.650945 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d0c2b9-c12f-40b4-a004-916bea47add5 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:05.652519 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 516/1011] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:55:05.658352 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfd669-7993-4ea2-8a3a-580632b1ce2b None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:05.658488 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de45d5-fbf0-4861-a130-e1636f3fea46 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:05.658661 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfd669-7993-4ea2-8a3a-580632b1ce2b None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:05.658911 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfd669-7993-4ea2-8a3a-580632b1ce2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:05.677193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-e6cfd669-7993-4ea2-8a3a-580632b1ce2b None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:05.678404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-e6cfd669-7993-4ea2-8a3a-580632b1ce2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:05.690769 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:05.697381 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed0583-47d4-4dc3-b2b0-ace983894950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:05.697993 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed0583-47d4-4dc3-b2b0-ace983894950 None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:05.698289 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed0583-47d4-4dc3-b2b0-ace983894950 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:05.698559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed0583-47d4-4dc3-b2b0-ace983894950 None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:05.699216 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed0583-47d4-4dc3-b2b0-ace983894950 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:05.700634 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed0583-47d4-4dc3-b2b0-ace983894950 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:05.730068 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 496/1012] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 14 01:55:05 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:55:05.760197 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 517/1013] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:05 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:55:06.029591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed0583-47d4-4dc3-b2b0-ace983894950 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:06.048941 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc62c1-7deb-4b55-916a-444b336e2f36 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:06.049456 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc62c1-7deb-4b55-916a-444b336e2f36 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:06.049456 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc62c1-7deb-4b55-916a-444b336e2f36 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:06.050977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 518/1014] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:55:06.055816 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03bce5b-4e5a-400c-b7d2-62e69469009e None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:06.056105 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03bce5b-4e5a-400c-b7d2-62e69469009e None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:06.056380 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03bce5b-4e5a-400c-b7d2-62e69469009e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:06.070253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:06.075616 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-c03bce5b-4e5a-400c-b7d2-62e69469009e None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:06.077352 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-c03bce5b-4e5a-400c-b7d2-62e69469009e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:06.144787 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed0583-47d4-4dc3-b2b0-ace983894950 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:06.148198 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 519/1015] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:06 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:55:06.180441 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3f3ac-a816-43b3-925f-cbff301e83c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:06.181339 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3f3ac-a816-43b3-925f-cbff301e83c4 None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:06.181799 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3f3ac-a816-43b3-925f-cbff301e83c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:06.182322 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3f3ac-a816-43b3-925f-cbff301e83c4 None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:06.183231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3f3ac-a816-43b3-925f-cbff301e83c4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:06.184846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3f3ac-a816-43b3-925f-cbff301e83c4 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:06.206023 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 497/1016] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 14 01:55:06 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:55:06.423528 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03bce5b-4e5a-400c-b7d2-62e69469009e None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:06.525867 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03bce5b-4e5a-400c-b7d2-62e69469009e None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:06.560640 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:06.567164 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2384c-5b59-4cbe-bf4c-f4fd37f65746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:06.568187 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2384c-5b59-4cbe-bf4c-f4fd37f65746 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:06.568469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2384c-5b59-4cbe-bf4c-f4fd37f65746 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:06.568740 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2384c-5b59-4cbe-bf4c-f4fd37f65746 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:06.569311 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d2384c-5b59-4cbe-bf4c-f4fd37f65746 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:06.570708 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d2384c-5b59-4cbe-bf4c-f4fd37f65746 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:06.580079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-37d2384c-5b59-4cbe-bf4c-f4fd37f65746 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 01:55:06.580079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:06.580079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:55:06.580079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:55:06.580079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:55:06.580079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:55:06.580079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:55:06.580079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:55:06.580079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 01:55:06.580079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:55:06.580079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:55:06.580079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:55:06.580079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:06.580079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:06.580079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:06.580079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:06.580079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:06.581084 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:06.581084 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:06.581084 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:06.581084 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:06.581084 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:06.581084 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:06.581084 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:55:06.581084 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:55:06.581084 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:55:06.581084 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:55:06.581084 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:06.581084 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:06.581084 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:06.581084 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:06.581084 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:06.581084 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:06.581957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:06.581957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:06.581957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:06.581957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:06.581957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:06.581957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:06.581957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:06.581957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:06.581957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:06.581957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:06.581957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:55:06.581957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:55:06.581957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 01:55:06.581957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:55:06.586040 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 520/1017] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:55:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:55:06.587428 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3f3ac-a816-43b3-925f-cbff301e83c4 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:06.631643 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:06.693697 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3f3ac-a816-43b3-925f-cbff301e83c4 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:06.732593 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de331e-af8c-4d78-a6dd-c858a21646c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:06.733200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de331e-af8c-4d78-a6dd-c858a21646c9 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:06.733479 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de331e-af8c-4d78-a6dd-c858a21646c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:06.733787 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de331e-af8c-4d78-a6dd-c858a21646c9 None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:06.734925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de331e-af8c-4d78-a6dd-c858a21646c9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:06.735226 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de331e-af8c-4d78-a6dd-c858a21646c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:06.736589 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de331e-af8c-4d78-a6dd-c858a21646c9 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:06.751074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 498/1018] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 14 01:55:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:06.751254 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2384c-5b59-4cbe-bf4c-f4fd37f65746 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:06.852189 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2384c-5b59-4cbe-bf4c-f4fd37f65746 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:06.885618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176e1fc-635f-4de3-9242-aaea00fd53ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:06.886282 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176e1fc-635f-4de3-9242-aaea00fd53ae None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:06.886564 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176e1fc-635f-4de3-9242-aaea00fd53ae None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:06.886837 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176e1fc-635f-4de3-9242-aaea00fd53ae None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:06.887460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176e1fc-635f-4de3-9242-aaea00fd53ae None admin] RBAC: Adding query filter params (type=compute) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:06.887764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176e1fc-635f-4de3-9242-aaea00fd53ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:06.889134 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176e1fc-635f-4de3-9242-aaea00fd53ae None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:06.898986 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 521/1019] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 14 01:55:06 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:55:06.900586 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de331e-af8c-4d78-a6dd-c858a21646c9 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:07.002288 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de331e-af8c-4d78-a6dd-c858a21646c9 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:07.009525 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:07.038508 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b0ff6-2dd5-4871-9dfb-e87c082e64b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:07.039156 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b0ff6-2dd5-4871-9dfb-e87c082e64b5 None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:07.039375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b0ff6-2dd5-4871-9dfb-e87c082e64b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:07.039636 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b0ff6-2dd5-4871-9dfb-e87c082e64b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:07.040305 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b0ff6-2dd5-4871-9dfb-e87c082e64b5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:07.041664 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b0ff6-2dd5-4871-9dfb-e87c082e64b5 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:07.061759 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 499/1020] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 14 01:55:06 2022] POST /identity/v3/endpoints => generated 334 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:55:07.938682 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df36a51-40b8-4a5c-986a-45e2cdc5312a None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:07.941114 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df36a51-40b8-4a5c-986a-45e2cdc5312a None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:07.941114 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df36a51-40b8-4a5c-986a-45e2cdc5312a None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:07.941114 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 522/1021] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:55:07.945961 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c32483-b042-401f-bbb0-2581d7c07170 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:07.946487 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c32483-b042-401f-bbb0-2581d7c07170 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:07.946652 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c32483-b042-401f-bbb0-2581d7c07170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:07.965104 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-a5c32483-b042-401f-bbb0-2581d7c07170 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:07.966319 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-a5c32483-b042-401f-bbb0-2581d7c07170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:08.045859 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 500/1022] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:55:08.311710 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df36a51-40b8-4a5c-986a-45e2cdc5312a None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:08.374040 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:08.401425 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d04cac-0775-4f9b-8cc3-f1515494ae60 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:08.401725 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d04cac-0775-4f9b-8cc3-f1515494ae60 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:08.402003 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d04cac-0775-4f9b-8cc3-f1515494ae60 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:08.403516 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 501/1023] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:55:08.408239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f155e-7cee-45af-8dd6-1b1b2ad4d9f5 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:08.408528 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f155e-7cee-45af-8dd6-1b1b2ad4d9f5 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:08.408800 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f155e-7cee-45af-8dd6-1b1b2ad4d9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:08.427345 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-9a8f155e-7cee-45af-8dd6-1b1b2ad4d9f5 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:08.427544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df36a51-40b8-4a5c-986a-45e2cdc5312a None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:08.428616 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-9a8f155e-7cee-45af-8dd6-1b1b2ad4d9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:08.463556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a285597-1552-4ea0-b7c6-a602e01b6b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:08.464297 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a285597-1552-4ea0-b7c6-a602e01b6b5f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:08.464577 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a285597-1552-4ea0-b7c6-a602e01b6b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:08.464847 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a285597-1552-4ea0-b7c6-a602e01b6b5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:08.465959 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a285597-1552-4ea0-b7c6-a602e01b6b5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:08.467428 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a285597-1552-4ea0-b7c6-a602e01b6b5f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:08.469842 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 523/1024] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:55:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:08.475169 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a285597-1552-4ea0-b7c6-a602e01b6b5f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:08.498021 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 502/1025] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:55:08.581005 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a285597-1552-4ea0-b7c6-a602e01b6b5f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:08.606725 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:08.615425 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2368d7-c6ba-444b-8160-3af9e7f07dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:08.616022 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2368d7-c6ba-444b-8160-3af9e7f07dfb None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:08.616304 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2368d7-c6ba-444b-8160-3af9e7f07dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:08.616576 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2368d7-c6ba-444b-8160-3af9e7f07dfb None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:08.617224 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2368d7-c6ba-444b-8160-3af9e7f07dfb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:08.618693 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2368d7-c6ba-444b-8160-3af9e7f07dfb None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:08.641131 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 524/1026] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 14 01:55:08 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:55:08.769077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f155e-7cee-45af-8dd6-1b1b2ad4d9f5 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:08.788656 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:08.875800 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f155e-7cee-45af-8dd6-1b1b2ad4d9f5 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:08.911186 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc2d5e-04e5-4474-b978-b5da5ee7aa91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:08.911824 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc2d5e-04e5-4474-b978-b5da5ee7aa91 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:08.912103 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc2d5e-04e5-4474-b978-b5da5ee7aa91 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:08.912369 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc2d5e-04e5-4474-b978-b5da5ee7aa91 None admin] PATH_INFO: `/v3/services/image` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:08.912942 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc2d5e-04e5-4474-b978-b5da5ee7aa91 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:08.914369 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc2d5e-04e5-4474-b978-b5da5ee7aa91 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:08.923654 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-6fbc2d5e-04e5-4474-b978-b5da5ee7aa91 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 01:55:08.923654 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:08.923654 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:55:08.923654 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:55:08.923654 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:55:08.923654 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:55:08.923654 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:55:08.923654 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:55:08.923654 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 01:55:08.923654 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:08.923654 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:55:08.923654 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:08.923654 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:08.923654 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:08.923654 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:08.923654 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:08.923654 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:08.924839 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:08.924839 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:08.924839 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:08.924839 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:08.924839 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:08.924839 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:08.924839 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:55:08.924839 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:55:08.924839 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:55:08.924839 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:55:08.924839 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:08.924839 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:08.924839 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:08.924839 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:08.924839 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:08.924839 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:08.925789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:08.925789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:08.925789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:08.925789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:08.925789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:08.925789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:08.925789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:08.925789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:08.925789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:08.925789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:08.925789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:55:08.925789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:55:08.925789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 01:55:08.925789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:08.925789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 503/1027] 10.222.0.15 () {56 vars in 1105 bytes} [Wed Sep 14 01:55:08 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:55:08.930833 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2368d7-c6ba-444b-8160-3af9e7f07dfb None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:09.031485 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2368d7-c6ba-444b-8160-3af9e7f07dfb None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:09.064523 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc578f0-7467-40f5-b0ae-b00a464d1c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:09.065130 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc578f0-7467-40f5-b0ae-b00a464d1c1f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:09.065406 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc578f0-7467-40f5-b0ae-b00a464d1c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:09.065672 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc578f0-7467-40f5-b0ae-b00a464d1c1f None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:09.066316 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc578f0-7467-40f5-b0ae-b00a464d1c1f None admin] RBAC: Adding query filter params (name=image) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:09.066616 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc578f0-7467-40f5-b0ae-b00a464d1c1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:09.067973 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc578f0-7467-40f5-b0ae-b00a464d1c1f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:09.073053 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 525/1028] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:55:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:09.079523 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc2d5e-04e5-4474-b978-b5da5ee7aa91 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:09.162607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:09.182333 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc2d5e-04e5-4474-b978-b5da5ee7aa91 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:09.219341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45120d6d-e8db-4a27-8ecd-2556dbcf9850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:09.219961 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45120d6d-e8db-4a27-8ecd-2556dbcf9850 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:09.220234 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45120d6d-e8db-4a27-8ecd-2556dbcf9850 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:09.220497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45120d6d-e8db-4a27-8ecd-2556dbcf9850 None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:09.221200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45120d6d-e8db-4a27-8ecd-2556dbcf9850 None admin] RBAC: Adding query filter params (type=image) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:09.221502 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45120d6d-e8db-4a27-8ecd-2556dbcf9850 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:09.223039 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45120d6d-e8db-4a27-8ecd-2556dbcf9850 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:09.233384 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 504/1029] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:55:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:09.234756 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc578f0-7467-40f5-b0ae-b00a464d1c1f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:09.286388 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:09.317313 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d14bca-fd92-4bd1-8ad3-cb4b0ec63f6c None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:09.317651 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d14bca-fd92-4bd1-8ad3-cb4b0ec63f6c None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:09.317875 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d14bca-fd92-4bd1-8ad3-cb4b0ec63f6c None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:09.319368 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 505/1030] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:55:09.324005 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125acc8-b50c-481f-b5af-29374ec54b43 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:09.324293 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125acc8-b50c-481f-b5af-29374ec54b43 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:09.324570 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125acc8-b50c-481f-b5af-29374ec54b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:09.343589 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-8125acc8-b50c-481f-b5af-29374ec54b43 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:09.344143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc578f0-7467-40f5-b0ae-b00a464d1c1f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:09.344756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-8125acc8-b50c-481f-b5af-29374ec54b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:09.385186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea05521-0b81-4125-b3f0-34f8ec561740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:09.386296 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea05521-0b81-4125-b3f0-34f8ec561740 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:09.386642 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea05521-0b81-4125-b3f0-34f8ec561740 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:09.386920 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea05521-0b81-4125-b3f0-34f8ec561740 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:09.387617 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea05521-0b81-4125-b3f0-34f8ec561740 None admin] RBAC: Adding query filter params (interface=public, service_id=17a4f2aae9fa4a5e9b291df07f51a732, region_id=RegionOne) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:09.387926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea05521-0b81-4125-b3f0-34f8ec561740 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:09.393425 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea05521-0b81-4125-b3f0-34f8ec561740 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:09.406370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 526/1031] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Sep 14 01:55:09 2022] GET /identity/v3/endpoints?service_id=17a4f2aae9fa4a5e9b291df07f51a732&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:09.407473 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea05521-0b81-4125-b3f0-34f8ec561740 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:09.434249 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 506/1032] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:55:09.511336 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea05521-0b81-4125-b3f0-34f8ec561740 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:09.524426 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:09.546350 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7704c-7573-4b33-8b89-55004d6aae33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:09.547546 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7704c-7573-4b33-8b89-55004d6aae33 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:09.547831 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7704c-7573-4b33-8b89-55004d6aae33 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:09.548102 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7704c-7573-4b33-8b89-55004d6aae33 None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:09.548756 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7704c-7573-4b33-8b89-55004d6aae33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:09.550162 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7704c-7573-4b33-8b89-55004d6aae33 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:09.562031 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 527/1033] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Sep 14 01:55:09 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:55:09.705410 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125acc8-b50c-481f-b5af-29374ec54b43 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:09.795595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:09.820581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125acc8-b50c-481f-b5af-29374ec54b43 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:09.853850 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a1bf7-2811-406e-be3f-efdb4a1a9d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:09.854509 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a1bf7-2811-406e-be3f-efdb4a1a9d08 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:09.854786 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a1bf7-2811-406e-be3f-efdb4a1a9d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:09.855056 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a1bf7-2811-406e-be3f-efdb4a1a9d08 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:09.865652 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a1bf7-2811-406e-be3f-efdb4a1a9d08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:55:09.870475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-0a7a1bf7-2811-406e-be3f-efdb4a1a9d08 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:55:09.870475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:09.870475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:09.870475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:09.870475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:09.870475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:09.870475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:09.870475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:09.870475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:09.870475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:09.870475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:09.870475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:09.870475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:55:09.870475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:55:09.870475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:55:09.870475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:55:09.871448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:55:09.871448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:55:09.871448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:55:09.871448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:55:09.871448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:09.871448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:09.871448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:09.871448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:09.871448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:09.871448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:09.871448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:09.871448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:09.871448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:09.871448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:09.871448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:09.871448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:09.872332 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:09.872332 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:09.872332 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:09.872332 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:09.872332 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:55:09.872332 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:55:09.872332 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:55:09.872332 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:55:09.872332 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:55:09.872332 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:55:09.872332 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:55:09.872332 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:09.872332 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 507/1034] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Sep 14 01:55:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:55:09.879575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7704c-7573-4b33-8b89-55004d6aae33 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:09.978871 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7704c-7573-4b33-8b89-55004d6aae33 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:10.018497 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee87bd-bbf8-4218-b064-dc10d1cf9143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:10.019106 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee87bd-bbf8-4218-b064-dc10d1cf9143 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:10.019381 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee87bd-bbf8-4218-b064-dc10d1cf9143 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:10.019649 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee87bd-bbf8-4218-b064-dc10d1cf9143 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:10.020307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee87bd-bbf8-4218-b064-dc10d1cf9143 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:10.020795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee87bd-bbf8-4218-b064-dc10d1cf9143 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:10.022246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee87bd-bbf8-4218-b064-dc10d1cf9143 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:10.036723 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 528/1035] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Sep 14 01:55:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:10.038549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a1bf7-2811-406e-be3f-efdb4a1a9d08 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:10.138749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a1bf7-2811-406e-be3f-efdb4a1a9d08 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:10.171702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64f0c0-d93b-49d8-93c2-f3619c63b29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:10.172318 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64f0c0-d93b-49d8-93c2-f3619c63b29c None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:10.172620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64f0c0-d93b-49d8-93c2-f3619c63b29c None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:10.172891 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64f0c0-d93b-49d8-93c2-f3619c63b29c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:10.175524 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:10.184358 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe64f0c0-d93b-49d8-93c2-f3619c63b29c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:10.185799 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe64f0c0-d93b-49d8-93c2-f3619c63b29c None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:10.200668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-fe64f0c0-d93b-49d8-93c2-f3619c63b29c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:55:10.200668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:10.200668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:10.200668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:10.200668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:10.200668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:10.200668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:10.200668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:10.200668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:10.200668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:10.200668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:10.200668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:10.200668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:55:10.200668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:55:10.200668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:55:10.200668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:55:10.201691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:10.201691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:10.201691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:10.201691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:10.201691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:10.201691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:10.201691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:10.201691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:10.201691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:10.201691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:10.201691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:10.201691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:10.201691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:10.201691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:10.201691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:10.201691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:10.201691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:55:10.207082 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:55:10.207082 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:55:10.207082 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:10.207082 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 508/1036] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:55:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:55:10.209213 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee87bd-bbf8-4218-b064-dc10d1cf9143 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:10.249667 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:10.310081 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee87bd-bbf8-4218-b064-dc10d1cf9143 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:10.348317 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc7c02-7ba2-4e2a-8ddf-03bffccf3648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:10.348924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc7c02-7ba2-4e2a-8ddf-03bffccf3648 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:10.349198 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc7c02-7ba2-4e2a-8ddf-03bffccf3648 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:10.349469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc7c02-7ba2-4e2a-8ddf-03bffccf3648 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:10.350137 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc7c02-7ba2-4e2a-8ddf-03bffccf3648 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:10.350437 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc7c02-7ba2-4e2a-8ddf-03bffccf3648 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:10.351795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc7c02-7ba2-4e2a-8ddf-03bffccf3648 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:10.367599 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 529/1037] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:10.368812 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64f0c0-d93b-49d8-93c2-f3619c63b29c None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:10.481376 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64f0c0-d93b-49d8-93c2-f3619c63b29c None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:10.514542 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf89069-9b3c-489a-9130-9e23e15997ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:10.515610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf89069-9b3c-489a-9130-9e23e15997ba None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:10.515893 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf89069-9b3c-489a-9130-9e23e15997ba None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:10.516162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf89069-9b3c-489a-9130-9e23e15997ba None admin] PATH_INFO: `/v3/users/nova` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:10.523757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf89069-9b3c-489a-9130-9e23e15997ba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:10.525190 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf89069-9b3c-489a-9130-9e23e15997ba None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:10.536714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-1cf89069-9b3c-489a-9130-9e23e15997ba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:55:10.536714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:10.536714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:10.536714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:10.536714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:10.536714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:10.536714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:10.536714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:10.536714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:10.536714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:10.536714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:10.536714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:10.536714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:55:10.536714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:55:10.536714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:55:10.536714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:55:10.537785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:10.537785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:10.537785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:55:10.537785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:55:10.537785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:55:10.537785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:55:10.537785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:10.537785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:10.537785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:10.537785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:10.537785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:10.537785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:10.537785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:10.537785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:10.537785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:10.537785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:10.537785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:10.537785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:10.538814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:10.538814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:10.538814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:55:10.538814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:55:10.538814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:55:10.538814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:55:10.538814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:55:10.538814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:55:10.538814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:55:10.538814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:10.538814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 509/1038] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Sep 14 01:55:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:55:10.545644 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc7c02-7ba2-4e2a-8ddf-03bffccf3648 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:10.640547 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:10.645777 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc7c02-7ba2-4e2a-8ddf-03bffccf3648 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:10.682538 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25222c-21b2-48cd-9244-cbb19213fd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:10.683148 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25222c-21b2-48cd-9244-cbb19213fd5b None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:10.683426 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25222c-21b2-48cd-9244-cbb19213fd5b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:10.683698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25222c-21b2-48cd-9244-cbb19213fd5b None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:10.684417 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25222c-21b2-48cd-9244-cbb19213fd5b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:10.684720 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25222c-21b2-48cd-9244-cbb19213fd5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:10.686166 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25222c-21b2-48cd-9244-cbb19213fd5b None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:10.704921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 530/1039] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Sep 14 01:55:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:55:10.710467 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf89069-9b3c-489a-9130-9e23e15997ba None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:10.762016 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:10.823975 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf89069-9b3c-489a-9130-9e23e15997ba None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:10.858309 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1c3ed-72bb-4beb-928e-13e17d28bbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:10.858924 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1c3ed-72bb-4beb-928e-13e17d28bbdc None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:10.859203 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1c3ed-72bb-4beb-928e-13e17d28bbdc None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:10.859473 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1c3ed-72bb-4beb-928e-13e17d28bbdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:10.893473 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a002aff-41eb-406e-921f-e4ed2f7d4b88 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:10.893794 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a002aff-41eb-406e-921f-e4ed2f7d4b88 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:10.894150 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a002aff-41eb-406e-921f-e4ed2f7d4b88 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:10.895415 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 531/1040] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:55:10.900246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79ea21-f5e1-4525-b0ca-9c165cd553ad None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:10.900527 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79ea21-f5e1-4525-b0ca-9c165cd553ad None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:10.900797 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79ea21-f5e1-4525-b0ca-9c165cd553ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:10.918405 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-df79ea21-f5e1-4525-b0ca-9c165cd553ad None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:10.919574 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-df79ea21-f5e1-4525-b0ca-9c165cd553ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:10.929609 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b1c3ed-72bb-4beb-928e-13e17d28bbdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:10.931462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b1c3ed-72bb-4beb-928e-13e17d28bbdc None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:10.981504 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:10.990469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 532/1041] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:55:11.038581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 510/1042] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:55:10 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:55:11.045129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79ea21-f5e1-4525-b0ca-9c165cd553ad None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:11.139918 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:11.155438 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79ea21-f5e1-4525-b0ca-9c165cd553ad None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:11.189325 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138b6b0-f6dc-4242-ba37-db3a3639031f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:11.190278 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138b6b0-f6dc-4242-ba37-db3a3639031f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:11.190794 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138b6b0-f6dc-4242-ba37-db3a3639031f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:11.191318 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138b6b0-f6dc-4242-ba37-db3a3639031f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:11.201260 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138b6b0-f6dc-4242-ba37-db3a3639031f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:11.203006 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138b6b0-f6dc-4242-ba37-db3a3639031f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:11.217509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-3138b6b0-f6dc-4242-ba37-db3a3639031f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:55:11.217509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:11.217509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:11.217509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:11.217509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:11.217509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:11.217509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:11.217509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:11.217509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:11.217509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:11.217509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:11.217509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:11.217509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:55:11.217509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:55:11.217509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:55:11.217509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:55:11.218610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:11.218610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:11.218610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:11.218610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:11.218610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:11.218610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:11.218610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:11.218610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:11.218610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:11.218610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:11.218610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:11.218610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:11.218610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:11.218610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:11.218610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:11.218610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:11.218610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:55:11.219557 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:55:11.219557 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:55:11.219557 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:55:11.219557 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 533/1043] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:55:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:55:11.225644 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1c3ed-72bb-4beb-928e-13e17d28bbdc None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:11.267995 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138b6b0-f6dc-4242-ba37-db3a3639031f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:11.335042 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1c3ed-72bb-4beb-928e-13e17d28bbdc None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:11.371690 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824a853-e52d-4a3c-8422-d43470667f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:11.372310 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824a853-e52d-4a3c-8422-d43470667f03 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:11.372668 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824a853-e52d-4a3c-8422-d43470667f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:11.372843 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824a853-e52d-4a3c-8422-d43470667f03 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:11.373476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824a853-e52d-4a3c-8422-d43470667f03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:11.374293 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824a853-e52d-4a3c-8422-d43470667f03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:11.375156 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138b6b0-f6dc-4242-ba37-db3a3639031f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:11.375265 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824a853-e52d-4a3c-8422-d43470667f03 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:11.394436 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 511/1044] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:11.397852 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824a853-e52d-4a3c-8422-d43470667f03 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:11.407132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:11.419969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801ccc8-4e96-4752-b060-32d678bdabe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:11.421019 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801ccc8-4e96-4752-b060-32d678bdabe0 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:11.421569 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801ccc8-4e96-4752-b060-32d678bdabe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:11.422033 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801ccc8-4e96-4752-b060-32d678bdabe0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:11.429441 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:11.435307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801ccc8-4e96-4752-b060-32d678bdabe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:55:11.442111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-7801ccc8-4e96-4752-b060-32d678bdabe0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:55:11.442111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:11.442111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:11.442111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:11.442111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:11.442111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:11.442111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:11.442111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:11.442111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:11.442111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:11.442111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:11.442111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:11.442111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:55:11.442111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:55:11.442111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:55:11.442111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:55:11.443072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:55:11.443072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:55:11.443072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:55:11.443072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:55:11.443072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:11.443072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:11.443072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:11.443072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:11.443072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:11.443072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:11.443072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:11.443072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:11.443072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:11.443072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:11.443072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:11.443072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:11.443935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:11.443935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:11.443935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:11.443935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:11.443935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:55:11.443935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:55:11.443935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:55:11.443935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:55:11.443935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:55:11.443935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:55:11.443935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:55:11.443935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:55:11.443935 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 534/1045] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:55:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:55:11.449008 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801ccc8-4e96-4752-b060-32d678bdabe0 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:11.514976 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824a853-e52d-4a3c-8422-d43470667f03 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:11.551545 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801ccc8-4e96-4752-b060-32d678bdabe0 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:11.556993 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249a102-239e-4426-ae65-f0dc0093aaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:11.557620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249a102-239e-4426-ae65-f0dc0093aaa0 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:11.557894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249a102-239e-4426-ae65-f0dc0093aaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:11.558204 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249a102-239e-4426-ae65-f0dc0093aaa0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:11.564990 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249a102-239e-4426-ae65-f0dc0093aaa0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:11.566601 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249a102-239e-4426-ae65-f0dc0093aaa0 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:11.578893 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-b249a102-239e-4426-ae65-f0dc0093aaa0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:55:11.578893 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:11.578893 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:11.578893 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:11.578893 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:11.578893 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:11.578893 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:11.578893 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:11.578893 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:11.578893 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:11.578893 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:11.578893 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:11.578893 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:55:11.578893 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:55:11.578893 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:55:11.579822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:55:11.579822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:11.579822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:11.579822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:11.579822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:11.579822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:11.579822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:11.579822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:11.579822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:11.579822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:11.579822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:11.579822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:11.579822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:11.579822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:11.579822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:11.579822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:11.579822 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:11.580767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:55:11.580767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:55:11.580767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:55:11.580767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:55:11.580767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:55:11.580767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:55:11.580767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:55:11.580767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:11.582559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 512/1046] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:55:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:55:11.585791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc74e7a-2a83-4434-b5cb-50ad81c639b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:11.586422 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc74e7a-2a83-4434-b5cb-50ad81c639b5 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:11.586700 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc74e7a-2a83-4434-b5cb-50ad81c639b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:11.586967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc74e7a-2a83-4434-b5cb-50ad81c639b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:11.587831 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc74e7a-2a83-4434-b5cb-50ad81c639b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:11.588134 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc74e7a-2a83-4434-b5cb-50ad81c639b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:11.588238 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249a102-239e-4426-ae65-f0dc0093aaa0 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:11.589604 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc74e7a-2a83-4434-b5cb-50ad81c639b5 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:11.599081 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 535/1047] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:55:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:55:11.605031 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc74e7a-2a83-4434-b5cb-50ad81c639b5 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:11.662921 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:11.673401 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:11.697965 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249a102-239e-4426-ae65-f0dc0093aaa0 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:11.717457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc74e7a-2a83-4434-b5cb-50ad81c639b5 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:11.735048 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba6868-579a-4726-87d7-d573b1370b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:11.735650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba6868-579a-4726-87d7-d573b1370b12 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:11.735928 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba6868-579a-4726-87d7-d573b1370b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:11.736195 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba6868-579a-4726-87d7-d573b1370b12 None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:11.736883 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba6868-579a-4726-87d7-d573b1370b12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:11.737180 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba6868-579a-4726-87d7-d573b1370b12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:11.738636 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba6868-579a-4726-87d7-d573b1370b12 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:11.750892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 513/1048] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:55:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:11.751038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac003f4-9db8-4dd0-9485-6147d13c7356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:11.751625 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac003f4-9db8-4dd0-9485-6147d13c7356 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:11.751900 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac003f4-9db8-4dd0-9485-6147d13c7356 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:11.752082 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5453115-280c-42c7-809a-3f73a710d00f None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:11.752187 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac003f4-9db8-4dd0-9485-6147d13c7356 None admin] PATH_INFO: `/v3/users/b13bf87b543e4d4980bae7542c1a0254` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:11.752345 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5453115-280c-42c7-809a-3f73a710d00f None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:11.752607 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5453115-280c-42c7-809a-3f73a710d00f None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:11.753794 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 514/1049] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:55:11.756898 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5453115-280c-42c7-809a-3f73a710d00f None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:11.769412 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac003f4-9db8-4dd0-9485-6147d13c7356 None admin] RBAC: Authorizing `identity:get_user(user_id=b13bf87b543e4d4980bae7542c1a0254)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:11.771047 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac003f4-9db8-4dd0-9485-6147d13c7356 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:11.772655 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 536/1050] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:55:11 2022] GET /identity/v3/users/b13bf87b543e4d4980bae7542c1a0254 => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:55:11.773790 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652bf33-a7bf-4381-ad8c-9f5e1ffc40b0 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:11.774093 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652bf33-a7bf-4381-ad8c-9f5e1ffc40b0 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:11.774361 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652bf33-a7bf-4381-ad8c-9f5e1ffc40b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:11.791291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-6652bf33-a7bf-4381-ad8c-9f5e1ffc40b0 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:11.792462 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-6652bf33-a7bf-4381-ad8c-9f5e1ffc40b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:11.855281 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5453115-280c-42c7-809a-3f73a710d00f None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:11.855916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 537/1051] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:55:11.856718 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6652bf33-a7bf-4381-ad8c-9f5e1ffc40b0 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:11.887098 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:11.896705 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcd335-d28b-40a7-aa2d-66c18f378584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:11.897328 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcd335-d28b-40a7-aa2d-66c18f378584 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:11.897603 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcd335-d28b-40a7-aa2d-66c18f378584 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:11.897868 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcd335-d28b-40a7-aa2d-66c18f378584 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:11.898611 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcd335-d28b-40a7-aa2d-66c18f378584 None admin] RBAC: Adding query filter params (role.id=4f2c7f12acdc45b48a2271b06f69a600, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=f721a3a4eca941989685448a87bb9a19) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:11.898908 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcd335-d28b-40a7-aa2d-66c18f378584 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:11.900309 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcd335-d28b-40a7-aa2d-66c18f378584 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:11.914096 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 515/1052] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:55:11 2022] GET /identity/v3/role_assignments?user.id=f721a3a4eca941989685448a87bb9a19&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=4f2c7f12acdc45b48a2271b06f69a600 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:11.918382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:11.974881 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6652bf33-a7bf-4381-ad8c-9f5e1ffc40b0 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:12.012177 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0ea06-d94e-4943-b5cf-3aa218a1865f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:12.012798 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0ea06-d94e-4943-b5cf-3aa218a1865f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:12.013081 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0ea06-d94e-4943-b5cf-3aa218a1865f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:12.013354 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0ea06-d94e-4943-b5cf-3aa218a1865f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:12.085256 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0ea06-d94e-4943-b5cf-3aa218a1865f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:12.087402 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0ea06-d94e-4943-b5cf-3aa218a1865f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:12.124330 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcd335-d28b-40a7-aa2d-66c18f378584 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:12.137838 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:12.198093 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 538/1053] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:55:11 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:55:12.200925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0ea06-d94e-4943-b5cf-3aa218a1865f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:12.235362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcd335-d28b-40a7-aa2d-66c18f378584 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:12.273666 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa377c-010e-4fa1-b1d6-c92f23e5b61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:12.274318 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa377c-010e-4fa1-b1d6-c92f23e5b61c None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:12.274591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa377c-010e-4fa1-b1d6-c92f23e5b61c None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:12.274858 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa377c-010e-4fa1-b1d6-c92f23e5b61c None admin] PATH_INFO: `/v3/services/image` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:12.275432 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa377c-010e-4fa1-b1d6-c92f23e5b61c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:12.276782 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa377c-010e-4fa1-b1d6-c92f23e5b61c None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:12.286112 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-c0fa377c-010e-4fa1-b1d6-c92f23e5b61c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 01:55:12.286112 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:12.286112 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:55:12.286112 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:55:12.286112 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:55:12.286112 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:55:12.286112 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:55:12.286112 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:55:12.286112 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 01:55:12.286112 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:12.286112 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:55:12.286112 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:12.286112 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:12.286112 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:12.286112 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:12.286112 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:12.286112 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:12.287123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:12.287123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:12.287123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:12.287123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:12.287123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:12.287123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:12.287123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:55:12.287123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:55:12.287123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:55:12.287123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:55:12.287123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:12.287123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:12.287123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:12.287123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:12.287123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:12.287123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:12.288001 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:12.288001 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:12.288001 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:12.288001 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:12.288001 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:12.288001 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:12.288001 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:12.288001 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:12.288001 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:12.288001 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:12.288001 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:55:12.288001 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:55:12.288001 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 01:55:12.288001 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:12.288001 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 516/1054] 10.222.0.15 () {56 vars in 1105 bytes} [Wed Sep 14 01:55:12 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:55:12.292898 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa377c-010e-4fa1-b1d6-c92f23e5b61c None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:12.317451 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0ea06-d94e-4943-b5cf-3aa218a1865f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:12.352186 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:12.357206 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e439e5-a729-42b3-b55c-196b018da3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:12.357835 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e439e5-a729-42b3-b55c-196b018da3fc None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:12.358155 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e439e5-a729-42b3-b55c-196b018da3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:12.358428 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e439e5-a729-42b3-b55c-196b018da3fc None admin] PATH_INFO: `/v3/projects/583f9ca7c28e4637bf9266badb4c6b56` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:12.359651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e439e5-a729-42b3-b55c-196b018da3fc None admin] RBAC: Authorizing `identity:get_project(project_id=583f9ca7c28e4637bf9266badb4c6b56)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:12.361099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e439e5-a729-42b3-b55c-196b018da3fc None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:12.366030 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 539/1055] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:55:12 2022] GET /identity/v3/projects/583f9ca7c28e4637bf9266badb4c6b56 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:55:12.368684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e439e5-a729-42b3-b55c-196b018da3fc None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:12.389284 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:12.412517 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa377c-010e-4fa1-b1d6-c92f23e5b61c None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:12.447709 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323e987-8966-4ee9-a661-59735b0149a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:12.448363 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323e987-8966-4ee9-a661-59735b0149a3 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:12.448585 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323e987-8966-4ee9-a661-59735b0149a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:12.448850 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323e987-8966-4ee9-a661-59735b0149a3 None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:12.449580 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323e987-8966-4ee9-a661-59735b0149a3 None admin] RBAC: Adding query filter params (name=image) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:12.449761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323e987-8966-4ee9-a661-59735b0149a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:12.451174 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323e987-8966-4ee9-a661-59735b0149a3 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:12.455922 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 517/1056] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:55:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:55:12.461458 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323e987-8966-4ee9-a661-59735b0149a3 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:12.496325 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e439e5-a729-42b3-b55c-196b018da3fc None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:12.542951 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b575ea-f6e0-469f-a8db-bb630be890a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:12.543702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b575ea-f6e0-469f-a8db-bb630be890a2 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:12.543980 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b575ea-f6e0-469f-a8db-bb630be890a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:12.544241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b575ea-f6e0-469f-a8db-bb630be890a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:12.544965 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b575ea-f6e0-469f-a8db-bb630be890a2 None admin] RBAC: Adding query filter params (role.id=aa420c2fb1ab4bb3b05b004a4d89779a, scope.project.id=583f9ca7c28e4637bf9266badb4c6b56, user.id=b13bf87b543e4d4980bae7542c1a0254) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:12.545381 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b575ea-f6e0-469f-a8db-bb630be890a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:12.547447 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b575ea-f6e0-469f-a8db-bb630be890a2 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:12.553008 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 540/1057] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:55:12 2022] GET /identity/v3/role_assignments?user.id=b13bf87b543e4d4980bae7542c1a0254&scope.project.id=583f9ca7c28e4637bf9266badb4c6b56&role.id=aa420c2fb1ab4bb3b05b004a4d89779a => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:12.587324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323e987-8966-4ee9-a661-59735b0149a3 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:12.609772 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:12.626093 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a95eef-733d-455e-9aa7-2456e2e0b338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:12.626716 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a95eef-733d-455e-9aa7-2456e2e0b338 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:12.626991 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a95eef-733d-455e-9aa7-2456e2e0b338 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:12.627259 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a95eef-733d-455e-9aa7-2456e2e0b338 None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:12.627900 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a95eef-733d-455e-9aa7-2456e2e0b338 None admin] RBAC: Adding query filter params (type=image) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:12.628196 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a95eef-733d-455e-9aa7-2456e2e0b338 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:12.629619 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a95eef-733d-455e-9aa7-2456e2e0b338 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:12.640780 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 518/1058] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:55:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:12.641864 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b575ea-f6e0-469f-a8db-bb630be890a2 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:12.729798 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:12.751191 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b575ea-f6e0-469f-a8db-bb630be890a2 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:12.790209 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910d42b-4de0-45ba-ae43-fbb04e1ef95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:12.790827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910d42b-4de0-45ba-ae43-fbb04e1ef95b None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:12.791102 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910d42b-4de0-45ba-ae43-fbb04e1ef95b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:12.791370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910d42b-4de0-45ba-ae43-fbb04e1ef95b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:12.792047 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910d42b-4de0-45ba-ae43-fbb04e1ef95b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:12.793411 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910d42b-4de0-45ba-ae43-fbb04e1ef95b None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:12.816571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 541/1059] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 14 01:55:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:55:14.142935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fc4ad-be26-4b4a-97ce-1cf16e57f79e None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:14.143485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fc4ad-be26-4b4a-97ce-1cf16e57f79e None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:14.143485 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fc4ad-be26-4b4a-97ce-1cf16e57f79e None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:14.144862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 519/1060] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:55:14.150136 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76325a7b-e810-4228-882e-9e0ab271852c None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:14.150524 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76325a7b-e810-4228-882e-9e0ab271852c None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:14.150707 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76325a7b-e810-4228-882e-9e0ab271852c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:14.168778 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-76325a7b-e810-4228-882e-9e0ab271852c None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:14.169961 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-76325a7b-e810-4228-882e-9e0ab271852c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:14.242363 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 542/1061] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:55:14.507391 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fc4ad-be26-4b4a-97ce-1cf16e57f79e None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:14.611072 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fc4ad-be26-4b4a-97ce-1cf16e57f79e None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:14.644777 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526a09c-f7ea-4f6d-bc6d-e5d6c60cdd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:14.645770 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526a09c-f7ea-4f6d-bc6d-e5d6c60cdd75 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:14.646290 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526a09c-f7ea-4f6d-bc6d-e5d6c60cdd75 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:14.646794 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526a09c-f7ea-4f6d-bc6d-e5d6c60cdd75 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:14.659571 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526a09c-f7ea-4f6d-bc6d-e5d6c60cdd75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:55:14.664954 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-2526a09c-f7ea-4f6d-bc6d-e5d6c60cdd75 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:55:14.664954 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:14.664954 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:14.664954 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:14.664954 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:14.664954 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:14.664954 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:14.664954 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:14.664954 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:14.664954 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:14.664954 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:14.664954 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:14.664954 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:55:14.664954 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:55:14.664954 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:55:14.664954 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:55:14.666140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:55:14.666140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:55:14.666140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:55:14.666140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:55:14.666140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:14.666140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:14.666140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:14.666140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:14.666140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:14.666140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:14.666140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:14.666140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:14.666140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:14.666140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:14.666140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:14.666140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:14.667003 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:14.667003 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:14.667003 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:14.667003 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:14.667003 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:55:14.667003 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:55:14.667003 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:55:14.667003 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:55:14.667003 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:55:14.667003 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:55:14.667003 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:55:14.667003 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:14.670207 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 520/1062] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Sep 14 01:55:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:55:14.674738 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76325a7b-e810-4228-882e-9e0ab271852c None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:14.736259 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:14.776376 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76325a7b-e810-4228-882e-9e0ab271852c None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:14.777136 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b8751-0313-4aad-b496-8d0799343cf5 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:14.777418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b8751-0313-4aad-b496-8d0799343cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:14.777689 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b8751-0313-4aad-b496-8d0799343cf5 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:14.779062 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 521/1063] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:55:14.784600 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd30e4-4586-4edb-82e7-cbba6e45a610 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:14.784884 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd30e4-4586-4edb-82e7-cbba6e45a610 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:14.785154 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd30e4-4586-4edb-82e7-cbba6e45a610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:14.803948 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-2cdd30e4-4586-4edb-82e7-cbba6e45a610 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:14.805149 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-2cdd30e4-4586-4edb-82e7-cbba6e45a610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:14.813955 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b2ac5-ed1b-4fd6-991e-0cbb68030993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:14.815218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b2ac5-ed1b-4fd6-991e-0cbb68030993 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:14.816022 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b2ac5-ed1b-4fd6-991e-0cbb68030993 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:14.816514 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b2ac5-ed1b-4fd6-991e-0cbb68030993 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:14.817376 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b2ac5-ed1b-4fd6-991e-0cbb68030993 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:14.817875 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b2ac5-ed1b-4fd6-991e-0cbb68030993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:14.819506 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b2ac5-ed1b-4fd6-991e-0cbb68030993 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:14.828716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 543/1064] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Sep 14 01:55:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:14.834268 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b2ac5-ed1b-4fd6-991e-0cbb68030993 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:14.847718 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:14.877398 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 522/1065] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:55:14.943852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b2ac5-ed1b-4fd6-991e-0cbb68030993 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:14.971485 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:14.984511 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9175ef-6c6f-4fa8-a7f2-a4fa8c92c809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:14.985161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9175ef-6c6f-4fa8-a7f2-a4fa8c92c809 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:14.985440 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9175ef-6c6f-4fa8-a7f2-a4fa8c92c809 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:14.985714 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9175ef-6c6f-4fa8-a7f2-a4fa8c92c809 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:14.996978 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9175ef-6c6f-4fa8-a7f2-a4fa8c92c809 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:14.998460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9175ef-6c6f-4fa8-a7f2-a4fa8c92c809 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:15.012926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-0b9175ef-6c6f-4fa8-a7f2-a4fa8c92c809 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:55:15.012926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:15.012926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:15.012926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:15.012926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:15.012926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:15.012926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:15.012926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:15.012926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:15.012926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:15.012926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:15.012926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:15.012926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:55:15.012926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:55:15.012926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:55:15.012926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:55:15.014151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:15.014151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:15.014151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:15.014151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:15.014151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:15.014151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:15.014151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:15.014151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:15.014151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:15.014151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:15.014151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:15.014151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:15.014151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:15.014151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:15.014151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:15.014151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:15.014151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:55:15.015649 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:55:15.015649 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:55:15.015649 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:55:15.015649 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 544/1066] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:55:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:55:15.021271 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd30e4-4586-4edb-82e7-cbba6e45a610 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:15.075458 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c4dfd-880a-4d2b-b0eb-eeb02f2577da None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:15.075728 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c4dfd-880a-4d2b-b0eb-eeb02f2577da None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:15.075991 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c4dfd-880a-4d2b-b0eb-eeb02f2577da None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:15.078005 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 545/1067] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:55:15.083176 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea488ac7-9171-4a2a-ae91-f982269cd04e None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:15.083458 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea488ac7-9171-4a2a-ae91-f982269cd04e None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:15.083730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea488ac7-9171-4a2a-ae91-f982269cd04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:15.107668 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-ea488ac7-9171-4a2a-ae91-f982269cd04e None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:15.109337 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-ea488ac7-9171-4a2a-ae91-f982269cd04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:15.133856 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd30e4-4586-4edb-82e7-cbba6e45a610 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:15.169327 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1e1e0-6468-4d3b-a0dd-cccfcf304760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:15.170038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1e1e0-6468-4d3b-a0dd-cccfcf304760 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:15.170223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1e1e0-6468-4d3b-a0dd-cccfcf304760 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:15.170471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1e1e0-6468-4d3b-a0dd-cccfcf304760 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:15.171111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1e1e0-6468-4d3b-a0dd-cccfcf304760 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:15.171927 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1e1e0-6468-4d3b-a0dd-cccfcf304760 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:15.173273 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1e1e0-6468-4d3b-a0dd-cccfcf304760 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:15.183132 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 546/1068] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:55:15.183973 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea488ac7-9171-4a2a-ae91-f982269cd04e None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:15.189205 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 523/1069] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:15.190167 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1e1e0-6468-4d3b-a0dd-cccfcf304760 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:15.226489 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:15.291098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea488ac7-9171-4a2a-ae91-f982269cd04e None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:15.292682 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1e1e0-6468-4d3b-a0dd-cccfcf304760 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:15.298808 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:15.328543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a84c5-d548-40fe-94a8-7259fec2d686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:15.328714 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7b6f2-ed93-45c9-a136-57b94530b8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:15.329098 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a84c5-d548-40fe-94a8-7259fec2d686 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:15.329214 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7b6f2-ed93-45c9-a136-57b94530b8eb None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:15.329378 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a84c5-d548-40fe-94a8-7259fec2d686 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:15.329483 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7b6f2-ed93-45c9-a136-57b94530b8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:15.329664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7b6f2-ed93-45c9-a136-57b94530b8eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:15.329772 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a84c5-d548-40fe-94a8-7259fec2d686 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:15.337364 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a84c5-d548-40fe-94a8-7259fec2d686 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:15.338812 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a84c5-d548-40fe-94a8-7259fec2d686 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:15.340382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7b6f2-ed93-45c9-a136-57b94530b8eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:55:15.345656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-11d7b6f2-ed93-45c9-a136-57b94530b8eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:55:15.345656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:15.345656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:15.345656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:15.345656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:15.345656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:15.345656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:15.345656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:15.345656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:15.345656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:15.345656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:15.345656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:15.345656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:55:15.345656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:55:15.345656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:55:15.345656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:55:15.346737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:55:15.346737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:55:15.346737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:55:15.346737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:55:15.346737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:15.346737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:15.346737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:15.346737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:15.346737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:15.346737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:15.346737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:15.346737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:15.346737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:15.346737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:15.346737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:15.346737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:15.347698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:15.347698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:15.347698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:15.347698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:15.347698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:55:15.347698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:55:15.347698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:55:15.347698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:55:15.347698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:55:15.347698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:55:15.347698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:55:15.347698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:55:15.347698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 547/1070] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:55:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:55:15.352720 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7b6f2-ed93-45c9-a136-57b94530b8eb None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:15.354111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-4a9a84c5-d548-40fe-94a8-7259fec2d686 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:55:15.354111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:15.354111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:15.354111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:15.354111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:15.354111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:15.354111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:15.354111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:15.354111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:15.354111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:15.354111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:15.354111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:15.354111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:55:15.354111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:55:15.354111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:55:15.354111 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:55:15.355064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:15.355064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:15.355064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:55:15.355064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:55:15.355064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:55:15.355064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:55:15.355064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:15.355064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:15.355064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:15.355064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:15.355064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:15.355064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:15.355064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:15.355064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:15.355064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:15.355064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:15.355064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:15.355064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:15.356050 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:15.356050 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:15.356050 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:55:15.356050 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:55:15.356050 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:55:15.356050 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:55:15.356050 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:55:15.356050 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:55:15.356050 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:55:15.356050 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:15.356050 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 524/1071] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Sep 14 01:55:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:55:15.361170 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a84c5-d548-40fe-94a8-7259fec2d686 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:15.469211 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7b6f2-ed93-45c9-a136-57b94530b8eb None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:15.470397 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:15.477569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a84c5-d548-40fe-94a8-7259fec2d686 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:15.502968 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff3997-7188-4188-bc36-d24658da9396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:15.503572 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff3997-7188-4188-bc36-d24658da9396 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:15.503848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff3997-7188-4188-bc36-d24658da9396 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:15.504117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff3997-7188-4188-bc36-d24658da9396 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:15.505015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff3997-7188-4188-bc36-d24658da9396 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:15.505313 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff3997-7188-4188-bc36-d24658da9396 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:15.506702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff3997-7188-4188-bc36-d24658da9396 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:15.511166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b334204-c907-4fc0-b334-49408cb5575e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:15.511771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b334204-c907-4fc0-b334-49408cb5575e None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:15.512050 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b334204-c907-4fc0-b334-49408cb5575e None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:15.512322 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b334204-c907-4fc0-b334-49408cb5575e None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:15.513067 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b334204-c907-4fc0-b334-49408cb5575e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:15.513370 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b334204-c907-4fc0-b334-49408cb5575e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:15.514542 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 548/1072] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:55:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:55:15.514802 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b334204-c907-4fc0-b334-49408cb5575e None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:15.515349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff3997-7188-4188-bc36-d24658da9396 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:15.540134 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 525/1073] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Sep 14 01:55:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:55:15.540134 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b334204-c907-4fc0-b334-49408cb5575e None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:15.542428 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:15.612878 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff3997-7188-4188-bc36-d24658da9396 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:15.634502 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b334204-c907-4fc0-b334-49408cb5575e None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:15.645710 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d057f-2c6e-45ec-aa9b-333f27885602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:15.646333 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d057f-2c6e-45ec-aa9b-333f27885602 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:15.646612 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d057f-2c6e-45ec-aa9b-333f27885602 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:15.646882 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d057f-2c6e-45ec-aa9b-333f27885602 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:15.668623 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c31d97-2122-4e9b-9b6f-58420ff2c355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:15.669245 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c31d97-2122-4e9b-9b6f-58420ff2c355 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:15.669546 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c31d97-2122-4e9b-9b6f-58420ff2c355 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:15.669812 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c31d97-2122-4e9b-9b6f-58420ff2c355 None admin] PATH_INFO: `/v3/users/b13bf87b543e4d4980bae7542c1a0254` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:15.670956 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c31d97-2122-4e9b-9b6f-58420ff2c355 None admin] RBAC: Authorizing `identity:get_user(user_id=b13bf87b543e4d4980bae7542c1a0254)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:15.672398 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c31d97-2122-4e9b-9b6f-58420ff2c355 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:15.673969 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 526/1074] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:55:15 2022] GET /identity/v3/users/b13bf87b543e4d4980bae7542c1a0254 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:15.674786 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c31d97-2122-4e9b-9b6f-58420ff2c355 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:15.696529 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:15.715073 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d057f-2c6e-45ec-aa9b-333f27885602 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:15.716529 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d057f-2c6e-45ec-aa9b-333f27885602 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:15.745964 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:15.775895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c31d97-2122-4e9b-9b6f-58420ff2c355 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:15.809552 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368454ca-9a6c-418b-b769-149c5c8fea83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:15.810200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368454ca-9a6c-418b-b769-149c5c8fea83 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:15.810476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368454ca-9a6c-418b-b769-149c5c8fea83 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:15.810752 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368454ca-9a6c-418b-b769-149c5c8fea83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:15.816816 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 549/1075] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:55:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:55:15.817675 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d057f-2c6e-45ec-aa9b-333f27885602 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:15.821797 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368454ca-9a6c-418b-b769-149c5c8fea83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:15.823276 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368454ca-9a6c-418b-b769-149c5c8fea83 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:15.838315 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-368454ca-9a6c-418b-b769-149c5c8fea83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:55:15.838315 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:15.838315 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:15.838315 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:15.838315 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:15.838315 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:15.838315 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:15.838315 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:15.838315 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:15.838315 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:15.838315 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:15.838315 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:15.838315 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:55:15.838315 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:55:15.838315 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:55:15.838315 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:55:15.839343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:15.839343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:15.839343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:15.839343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:15.839343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:15.839343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:15.839343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:15.839343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:15.839343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:15.839343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:15.839343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:15.839343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:15.839343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:15.839343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:15.839343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:15.839343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:15.839343 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:55:15.840702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:55:15.840702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:55:15.840702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:15.840702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 527/1076] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:55:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:55:15.840702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368454ca-9a6c-418b-b769-149c5c8fea83 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:15.907610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:15.921737 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d057f-2c6e-45ec-aa9b-333f27885602 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:15.937876 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368454ca-9a6c-418b-b769-149c5c8fea83 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:15.947312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:15.955659 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1a5c3-2ba6-4d1d-ab8d-8ab07f7f441f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:15.956280 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1a5c3-2ba6-4d1d-ab8d-8ab07f7f441f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:15.956557 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1a5c3-2ba6-4d1d-ab8d-8ab07f7f441f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:15.956827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1a5c3-2ba6-4d1d-ab8d-8ab07f7f441f None admin] PATH_INFO: `/v3/projects/583f9ca7c28e4637bf9266badb4c6b56` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:15.957917 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1a5c3-2ba6-4d1d-ab8d-8ab07f7f441f None admin] RBAC: Authorizing `identity:get_project(project_id=583f9ca7c28e4637bf9266badb4c6b56)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:15.959367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1a5c3-2ba6-4d1d-ab8d-8ab07f7f441f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:15.960963 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 550/1077] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:55:15 2022] GET /identity/v3/projects/583f9ca7c28e4637bf9266badb4c6b56 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:15.961713 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1a5c3-2ba6-4d1d-ab8d-8ab07f7f441f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:15.971467 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0efd0-f71a-4231-a511-2e3f98ddb989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:15.972090 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0efd0-f71a-4231-a511-2e3f98ddb989 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:15.972366 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0efd0-f71a-4231-a511-2e3f98ddb989 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:15.972631 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0efd0-f71a-4231-a511-2e3f98ddb989 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:15.982982 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0efd0-f71a-4231-a511-2e3f98ddb989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:15.984461 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0efd0-f71a-4231-a511-2e3f98ddb989 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:15.998587 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-7bc0efd0-f71a-4231-a511-2e3f98ddb989 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:55:15.998587 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:15.998587 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:15.998587 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:15.998587 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:15.998587 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:15.998587 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:15.998587 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:15.998587 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:15.998587 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:15.998587 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:15.998587 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:15.998587 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:55:15.998587 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:55:15.998587 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:55:15.998587 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:55:16.000257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:16.000257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:16.000257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:16.000257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:16.000257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:16.000257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:16.000257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:16.000257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:16.000257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:16.000257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:16.000257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:16.000257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:16.000257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:16.000257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:16.000257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:16.000257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:16.000257 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:55:16.001180 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:55:16.001180 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:55:16.001180 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:16.001180 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 528/1078] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:55:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:55:16.001180 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0efd0-f71a-4231-a511-2e3f98ddb989 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:16.064871 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1a5c3-2ba6-4d1d-ab8d-8ab07f7f441f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:16.097765 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea9ba6-2f98-4cc1-916a-9ace0f4d836f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:16.098396 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea9ba6-2f98-4cc1-916a-9ace0f4d836f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:16.098681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea9ba6-2f98-4cc1-916a-9ace0f4d836f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:16.098957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea9ba6-2f98-4cc1-916a-9ace0f4d836f None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:16.099590 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea9ba6-2f98-4cc1-916a-9ace0f4d836f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:16.099894 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea9ba6-2f98-4cc1-916a-9ace0f4d836f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:16.101261 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea9ba6-2f98-4cc1-916a-9ace0f4d836f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:16.101636 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0efd0-f71a-4231-a511-2e3f98ddb989 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:16.112708 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 551/1079] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:16.113574 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea9ba6-2f98-4cc1-916a-9ace0f4d836f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:16.124826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:16.135854 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1ce1e-25a7-48f7-bb26-d9307c7db465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:16.136503 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1ce1e-25a7-48f7-bb26-d9307c7db465 None admin] REQUEST_METHOD: `PUT` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:16.136778 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1ce1e-25a7-48f7-bb26-d9307c7db465 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:16.137044 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1ce1e-25a7-48f7-bb26-d9307c7db465 None admin] PATH_INFO: `/v3/projects/583f9ca7c28e4637bf9266badb4c6b56/users/b13bf87b543e4d4980bae7542c1a0254/roles/aa420c2fb1ab4bb3b05b004a4d89779a` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:16.138958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1ce1e-25a7-48f7-bb26-d9307c7db465 None admin] RBAC: Authorizing `identity:create_grant(project_id=583f9ca7c28e4637bf9266badb4c6b56, user_id=b13bf87b543e4d4980bae7542c1a0254, role_id=aa420c2fb1ab4bb3b05b004a4d89779a)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:16.140418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1ce1e-25a7-48f7-bb26-d9307c7db465 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:16.152717 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 529/1080] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:55:16 2022] PUT /identity/v3/projects/583f9ca7c28e4637bf9266badb4c6b56/users/b13bf87b543e4d4980bae7542c1a0254/roles/aa420c2fb1ab4bb3b05b004a4d89779a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:55:16.153495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1ce1e-25a7-48f7-bb26-d9307c7db465 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:16.182741 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:16.235069 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea9ba6-2f98-4cc1-916a-9ace0f4d836f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:16.267117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1ce1e-25a7-48f7-bb26-d9307c7db465 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:16.274706 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b000bf-dd57-432f-9ae6-a9f6e725986b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:16.275343 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b000bf-dd57-432f-9ae6-a9f6e725986b None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:16.276187 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b000bf-dd57-432f-9ae6-a9f6e725986b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:16.276473 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b000bf-dd57-432f-9ae6-a9f6e725986b None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:16.277134 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b000bf-dd57-432f-9ae6-a9f6e725986b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:16.277439 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b000bf-dd57-432f-9ae6-a9f6e725986b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:16.279362 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b000bf-dd57-432f-9ae6-a9f6e725986b None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:16.302089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 552/1081] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:55:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:16.305595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33752a20-0819-4dd9-ac39-b9a2df9ab3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:16.306285 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33752a20-0819-4dd9-ac39-b9a2df9ab3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:16.306560 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33752a20-0819-4dd9-ac39-b9a2df9ab3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:16.306826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33752a20-0819-4dd9-ac39-b9a2df9ab3c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:16.314973 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33752a20-0819-4dd9-ac39-b9a2df9ab3c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:16.316449 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33752a20-0819-4dd9-ac39-b9a2df9ab3c0 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:16.330481 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-33752a20-0819-4dd9-ac39-b9a2df9ab3c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:55:16.330481 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:16.330481 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:16.330481 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:16.330481 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:16.330481 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:16.330481 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:16.330481 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:16.330481 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:16.330481 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:16.330481 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:16.330481 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:16.330481 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:55:16.330481 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:55:16.330481 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:55:16.331501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:55:16.331501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:16.331501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:16.331501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:16.331501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:16.331501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:16.331501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:16.331501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:16.331501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:16.331501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:16.331501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:16.331501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:16.331501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:16.331501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:16.331501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:16.331501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:16.331501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:16.332448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:55:16.332448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:55:16.332448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:55:16.332448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:55:16.332448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:55:16.332448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:55:16.332448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:55:16.332448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:16.332448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 530/1082] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:55:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:55:16.338699 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b000bf-dd57-432f-9ae6-a9f6e725986b None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:16.452387 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b000bf-dd57-432f-9ae6-a9f6e725986b None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:16.470015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:16.493635 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104ce48-9e61-494c-90fa-300120aec170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:16.494448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104ce48-9e61-494c-90fa-300120aec170 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:16.494726 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104ce48-9e61-494c-90fa-300120aec170 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:16.494994 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104ce48-9e61-494c-90fa-300120aec170 None admin] PATH_INFO: `/v3/projects` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:16.495706 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104ce48-9e61-494c-90fa-300120aec170 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:16.496007 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104ce48-9e61-494c-90fa-300120aec170 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:16.497436 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104ce48-9e61-494c-90fa-300120aec170 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:16.510583 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 553/1083] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:55:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:16.516202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33752a20-0819-4dd9-ac39-b9a2df9ab3c0 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:16.564838 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:16.620944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33752a20-0819-4dd9-ac39-b9a2df9ab3c0 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:16.656616 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd03a2-39c9-49af-8198-ec47b601a6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:16.657279 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd03a2-39c9-49af-8198-ec47b601a6e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:16.657559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd03a2-39c9-49af-8198-ec47b601a6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:16.657830 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd03a2-39c9-49af-8198-ec47b601a6e1 None admin] PATH_INFO: `/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/f721a3a4eca941989685448a87bb9a19/roles/4f2c7f12acdc45b48a2271b06f69a600` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:16.659996 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd03a2-39c9-49af-8198-ec47b601a6e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e04b16dba1c4de79dc6b1a5e16ed907, user_id=f721a3a4eca941989685448a87bb9a19, role_id=4f2c7f12acdc45b48a2271b06f69a600)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:16.661427 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd03a2-39c9-49af-8198-ec47b601a6e1 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:16.675596 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 531/1084] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:55:16 2022] PUT /identity/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/f721a3a4eca941989685448a87bb9a19/roles/4f2c7f12acdc45b48a2271b06f69a600 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:55:18.315752 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb44e57-64d3-457b-95d5-6920c3308374 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:18.316301 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb44e57-64d3-457b-95d5-6920c3308374 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:18.316301 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb44e57-64d3-457b-95d5-6920c3308374 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:18.317613 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 554/1085] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:55:18.322772 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb028d7a-0fa8-431b-a687-c6f5a54ec592 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:18.323195 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb028d7a-0fa8-431b-a687-c6f5a54ec592 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:18.323309 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb028d7a-0fa8-431b-a687-c6f5a54ec592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:18.341768 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-cb028d7a-0fa8-431b-a687-c6f5a54ec592 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:18.343471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-cb028d7a-0fa8-431b-a687-c6f5a54ec592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:18.408916 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 532/1086] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:55:18.675283 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb44e57-64d3-457b-95d5-6920c3308374 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:18.782544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb44e57-64d3-457b-95d5-6920c3308374 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:18.815011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14672bb1-8fc6-4d5a-9987-22795197afa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:18.815626 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14672bb1-8fc6-4d5a-9987-22795197afa0 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:18.815905 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14672bb1-8fc6-4d5a-9987-22795197afa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:18.816172 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14672bb1-8fc6-4d5a-9987-22795197afa0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:18.826370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14672bb1-8fc6-4d5a-9987-22795197afa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:55:18.830233 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfb856-ef26-4c79-8098-fd90863bf8b6 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:18.830527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfb856-ef26-4c79-8098-fd90863bf8b6 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:18.830802 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfb856-ef26-4c79-8098-fd90863bf8b6 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:18.831621 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-14672bb1-8fc6-4d5a-9987-22795197afa0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:55:18.831621 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:18.831621 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:18.831621 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:18.831621 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:18.831621 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:18.831621 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:18.831621 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:18.831621 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:18.831621 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:18.831621 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:18.831621 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:18.831621 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:55:18.831621 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:55:18.831621 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:55:18.831621 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:55:18.832607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:55:18.832607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:55:18.832607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:55:18.832607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:55:18.832607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:18.832607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:18.832607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:18.832607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:18.832607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:18.832607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:18.832607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:18.832607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:18.832607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:18.832607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:18.832607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:18.832607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:18.833472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:18.833472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:18.833472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:18.833472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:18.833472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:55:18.833472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:55:18.833472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:55:18.833472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:55:18.833472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:55:18.833472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:55:18.833472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:55:18.833472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:55:18.834171 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 533/1087] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:55:18.834245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 555/1088] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:55:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:55:18.837023 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab07b1f-87b3-4830-aa71-196a3159133b None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:18.837417 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab07b1f-87b3-4830-aa71-196a3159133b None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:18.837695 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab07b1f-87b3-4830-aa71-196a3159133b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:18.838409 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14672bb1-8fc6-4d5a-9987-22795197afa0 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:18.855715 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-5ab07b1f-87b3-4830-aa71-196a3159133b None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:18.856879 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-5ab07b1f-87b3-4830-aa71-196a3159133b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:18.871769 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:18.882929 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:18.927909 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 534/1089] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:55:18.937287 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14672bb1-8fc6-4d5a-9987-22795197afa0 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:18.969153 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103c2e4-f81f-4c94-8193-08bf37e2521a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:18.969740 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103c2e4-f81f-4c94-8193-08bf37e2521a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:18.970044 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103c2e4-f81f-4c94-8193-08bf37e2521a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:18.970316 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103c2e4-f81f-4c94-8193-08bf37e2521a None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:18.970957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103c2e4-f81f-4c94-8193-08bf37e2521a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:18.971255 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103c2e4-f81f-4c94-8193-08bf37e2521a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:18.972605 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103c2e4-f81f-4c94-8193-08bf37e2521a None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:18.980643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 556/1090] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:55:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:18.986494 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab07b1f-87b3-4830-aa71-196a3159133b None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:19.082091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab07b1f-87b3-4830-aa71-196a3159133b None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:19.113964 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37707c1-2b91-4a0b-868e-eedfa6bad4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:19.114602 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37707c1-2b91-4a0b-868e-eedfa6bad4d1 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:19.114874 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37707c1-2b91-4a0b-868e-eedfa6bad4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:19.115137 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37707c1-2b91-4a0b-868e-eedfa6bad4d1 None admin] PATH_INFO: `/v3/users/b13bf87b543e4d4980bae7542c1a0254` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:19.116237 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37707c1-2b91-4a0b-868e-eedfa6bad4d1 None admin] RBAC: Authorizing `identity:get_user(user_id=b13bf87b543e4d4980bae7542c1a0254)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:19.117642 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37707c1-2b91-4a0b-868e-eedfa6bad4d1 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:19.119209 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 535/1091] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:55:18 2022] GET /identity/v3/users/b13bf87b543e4d4980bae7542c1a0254 => generated 285 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:19.125386 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103c2e4-f81f-4c94-8193-08bf37e2521a None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:19.195272 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37707c1-2b91-4a0b-868e-eedfa6bad4d1 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:19.221116 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:19.225089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103c2e4-f81f-4c94-8193-08bf37e2521a None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:19.226690 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:19.259472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46197bc0-1cb2-4a0a-b453-25721aeefaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:19.260072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46197bc0-1cb2-4a0a-b453-25721aeefaf1 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:19.260346 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46197bc0-1cb2-4a0a-b453-25721aeefaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:19.260610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46197bc0-1cb2-4a0a-b453-25721aeefaf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:19.297177 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37707c1-2b91-4a0b-868e-eedfa6bad4d1 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:19.326875 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46197bc0-1cb2-4a0a-b453-25721aeefaf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:19.328377 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46197bc0-1cb2-4a0a-b453-25721aeefaf1 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:19.330861 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49a2ca-3488-4fac-85ac-c889a5f05c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:19.331482 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49a2ca-3488-4fac-85ac-c889a5f05c54 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:19.331758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49a2ca-3488-4fac-85ac-c889a5f05c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:19.332026 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49a2ca-3488-4fac-85ac-c889a5f05c54 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:19.341850 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd49a2ca-3488-4fac-85ac-c889a5f05c54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:55:19.346862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-cd49a2ca-3488-4fac-85ac-c889a5f05c54 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:55:19.346862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:19.346862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:19.346862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:19.346862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:19.346862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:19.346862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:19.346862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:19.346862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:19.346862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:19.346862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:19.346862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:19.346862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:55:19.346862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:55:19.346862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:55:19.346862 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:55:19.347983 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:55:19.347983 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:55:19.347983 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:55:19.347983 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:55:19.347983 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:19.347983 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:19.347983 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:19.347983 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:19.347983 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:19.347983 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:19.347983 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:19.347983 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:19.347983 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:19.347983 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:19.347983 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:19.347983 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:19.348883 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:19.348883 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:19.348883 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:19.348883 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:19.348883 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:55:19.348883 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:55:19.348883 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:55:19.348883 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:55:19.348883 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:55:19.348883 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:55:19.348883 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:55:19.348883 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:19.348883 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 536/1092] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Sep 14 01:55:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:55:19.353877 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49a2ca-3488-4fac-85ac-c889a5f05c54 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:19.422696 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:19.428051 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 557/1093] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:55:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 14 01:55:19.433809 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46197bc0-1cb2-4a0a-b453-25721aeefaf1 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:19.445555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:19.454338 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49a2ca-3488-4fac-85ac-c889a5f05c54 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:19.488937 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0df6a1-b71f-462c-9ff1-07215aedf9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:19.489537 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0df6a1-b71f-462c-9ff1-07215aedf9a1 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:19.489811 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0df6a1-b71f-462c-9ff1-07215aedf9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:19.490101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0df6a1-b71f-462c-9ff1-07215aedf9a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:19.490756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0df6a1-b71f-462c-9ff1-07215aedf9a1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:19.491049 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0df6a1-b71f-462c-9ff1-07215aedf9a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:19.492398 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0df6a1-b71f-462c-9ff1-07215aedf9a1 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:19.500712 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 537/1094] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Sep 14 01:55:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:19.506748 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0df6a1-b71f-462c-9ff1-07215aedf9a1 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:19.531121 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46197bc0-1cb2-4a0a-b453-25721aeefaf1 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:19.565805 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6588fb1-7775-4cf2-9cf4-13ee4b1a55c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:19.566577 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6588fb1-7775-4cf2-9cf4-13ee4b1a55c8 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:19.566925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6588fb1-7775-4cf2-9cf4-13ee4b1a55c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:19.567204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6588fb1-7775-4cf2-9cf4-13ee4b1a55c8 None admin] PATH_INFO: `/v3/projects/583f9ca7c28e4637bf9266badb4c6b56` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:19.568302 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6588fb1-7775-4cf2-9cf4-13ee4b1a55c8 None admin] RBAC: Authorizing `identity:get_project(project_id=583f9ca7c28e4637bf9266badb4c6b56)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:19.569729 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6588fb1-7775-4cf2-9cf4-13ee4b1a55c8 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:19.571365 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 558/1095] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:55:19 2022] GET /identity/v3/projects/583f9ca7c28e4637bf9266badb4c6b56 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:55:19.577329 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6588fb1-7775-4cf2-9cf4-13ee4b1a55c8 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:19.603355 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0df6a1-b71f-462c-9ff1-07215aedf9a1 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:19.637328 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa23ce9-b348-49c1-9fbc-dc85be0f1599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:19.637948 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa23ce9-b348-49c1-9fbc-dc85be0f1599 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:19.638255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa23ce9-b348-49c1-9fbc-dc85be0f1599 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:19.638530 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa23ce9-b348-49c1-9fbc-dc85be0f1599 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:19.641141 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:19.649142 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa23ce9-b348-49c1-9fbc-dc85be0f1599 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:19.650622 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa23ce9-b348-49c1-9fbc-dc85be0f1599 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:19.665476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-3fa23ce9-b348-49c1-9fbc-dc85be0f1599 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:55:19.665476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:19.665476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:19.665476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:19.665476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:19.665476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:19.665476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:19.665476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:19.665476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:19.665476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:19.665476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:19.665476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:19.665476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:55:19.665476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:55:19.665476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:55:19.665476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:55:19.666650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:19.666650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:19.666650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:19.666650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:19.666650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:19.666650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:19.666650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:19.666650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:19.666650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:19.666650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:19.666650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:19.666650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:19.666650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:19.666650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:19.666650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:19.666650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:19.666650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:55:19.667626 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:55:19.667626 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:55:19.667626 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:19.667626 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 538/1096] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:55:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:55:19.672825 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa23ce9-b348-49c1-9fbc-dc85be0f1599 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:19.674996 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6588fb1-7775-4cf2-9cf4-13ee4b1a55c8 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:19.681495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:19.708763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b847a-4f52-4949-8c0b-741792cfc81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:19.709294 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b847a-4f52-4949-8c0b-741792cfc81a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:19.709573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b847a-4f52-4949-8c0b-741792cfc81a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:19.709843 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b847a-4f52-4949-8c0b-741792cfc81a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:19.710583 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b847a-4f52-4949-8c0b-741792cfc81a None admin] RBAC: Adding query filter params (role.id=aa420c2fb1ab4bb3b05b004a4d89779a, scope.project.id=583f9ca7c28e4637bf9266badb4c6b56, user.id=b13bf87b543e4d4980bae7542c1a0254) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:19.710884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b847a-4f52-4949-8c0b-741792cfc81a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:19.712293 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b847a-4f52-4949-8c0b-741792cfc81a None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:19.718419 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 559/1097] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:55:19 2022] GET /identity/v3/role_assignments?user.id=b13bf87b543e4d4980bae7542c1a0254&scope.project.id=583f9ca7c28e4637bf9266badb4c6b56&role.id=aa420c2fb1ab4bb3b05b004a4d89779a => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:55:19.773583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa23ce9-b348-49c1-9fbc-dc85be0f1599 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:19.810050 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd2fba-5681-4439-a791-9585666dd8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:19.810850 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd2fba-5681-4439-a791-9585666dd8fd None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:19.811191 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd2fba-5681-4439-a791-9585666dd8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:19.811455 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd2fba-5681-4439-a791-9585666dd8fd None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:19.812154 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd2fba-5681-4439-a791-9585666dd8fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:19.812528 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd2fba-5681-4439-a791-9585666dd8fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:19.813901 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd2fba-5681-4439-a791-9585666dd8fd None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:19.826644 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 539/1098] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:55:19.832650 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b847a-4f52-4949-8c0b-741792cfc81a None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:19.938253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b847a-4f52-4949-8c0b-741792cfc81a None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:19.965143 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:19.971642 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c71bb1-f871-4d5a-ae1a-be43b7e30f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:19.972263 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c71bb1-f871-4d5a-ae1a-be43b7e30f89 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:19.972538 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c71bb1-f871-4d5a-ae1a-be43b7e30f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:19.972808 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c71bb1-f871-4d5a-ae1a-be43b7e30f89 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:19.980122 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c71bb1-f871-4d5a-ae1a-be43b7e30f89 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:19.981542 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c71bb1-f871-4d5a-ae1a-be43b7e30f89 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:19.992349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-c8c71bb1-f871-4d5a-ae1a-be43b7e30f89 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:55:19.992349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:19.992349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:19.992349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:19.992349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:19.992349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:19.992349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:19.992349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:19.992349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:19.992349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:19.992349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:19.992349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:19.992349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:55:19.992349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:55:19.992349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:55:19.992349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:55:19.993471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:19.993471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:19.993471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:55:19.993471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:55:19.993471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:55:19.993471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:55:19.993471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:19.993471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:19.993471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:19.993471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:19.993471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:19.993471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:19.993471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:19.993471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:19.993471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:19.993471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:19.993471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:19.993471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:19.994823 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:19.994823 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:19.994823 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:55:19.994823 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:55:19.994823 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:55:19.994823 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:55:19.994823 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:55:19.994823 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:55:19.994823 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:55:19.994823 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:55:19.994823 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 560/1099] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Sep 14 01:55:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:55:19.999804 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd2fba-5681-4439-a791-9585666dd8fd None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:20.063136 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:20.097470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd2fba-5681-4439-a791-9585666dd8fd None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:20.131354 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2588bf-258e-4c2c-a020-d5f216e76c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:20.131947 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2588bf-258e-4c2c-a020-d5f216e76c3c None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:20.132222 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2588bf-258e-4c2c-a020-d5f216e76c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:20.132492 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2588bf-258e-4c2c-a020-d5f216e76c3c None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:20.133201 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2588bf-258e-4c2c-a020-d5f216e76c3c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:20.133498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2588bf-258e-4c2c-a020-d5f216e76c3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:20.134938 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2588bf-258e-4c2c-a020-d5f216e76c3c None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:20.151563 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 540/1100] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Sep 14 01:55:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:20.157138 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c71bb1-f871-4d5a-ae1a-be43b7e30f89 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:20.255315 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c71bb1-f871-4d5a-ae1a-be43b7e30f89 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:20.287167 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0abaf6-4a93-4dc5-92a3-dddb339f3e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:20.287769 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0abaf6-4a93-4dc5-92a3-dddb339f3e82 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:20.288047 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0abaf6-4a93-4dc5-92a3-dddb339f3e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:20.288317 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0abaf6-4a93-4dc5-92a3-dddb339f3e82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:20.343570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:20.354200 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0abaf6-4a93-4dc5-92a3-dddb339f3e82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:20.355634 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0abaf6-4a93-4dc5-92a3-dddb339f3e82 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:20.451976 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 561/1101] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:55:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:55:20.457521 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2588bf-258e-4c2c-a020-d5f216e76c3c None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:20.553350 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2588bf-258e-4c2c-a020-d5f216e76c3c None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:20.575597 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:20.586756 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04733787-ceb0-4c12-992c-a1c133e7f8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:20.587372 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04733787-ceb0-4c12-992c-a1c133e7f8b1 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:20.587647 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04733787-ceb0-4c12-992c-a1c133e7f8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:20.587912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04733787-ceb0-4c12-992c-a1c133e7f8b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:20.597603 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04733787-ceb0-4c12-992c-a1c133e7f8b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:20.599166 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04733787-ceb0-4c12-992c-a1c133e7f8b1 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:20.614094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-04733787-ceb0-4c12-992c-a1c133e7f8b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:55:20.614094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:20.614094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:20.614094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:20.614094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:20.614094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:20.614094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:20.614094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:20.614094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:20.614094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:20.614094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:20.614094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:20.614094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:55:20.614094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:55:20.614094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:55:20.614094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:55:20.615324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:20.615324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:20.615324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:20.615324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:20.615324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:20.615324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:20.615324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:20.615324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:20.615324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:20.615324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:20.615324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:20.615324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:20.615324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:20.615324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:20.615324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:20.615324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:20.615324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:55:20.616648 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:55:20.616648 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:55:20.616648 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:20.616648 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 541/1102] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:55:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:55:20.621233 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0abaf6-4a93-4dc5-92a3-dddb339f3e82 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:20.708493 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:20.718216 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0abaf6-4a93-4dc5-92a3-dddb339f3e82 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:20.750820 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494cb761-b101-493c-97dc-76b99c9d942b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:20.751409 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494cb761-b101-493c-97dc-76b99c9d942b None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:20.751683 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494cb761-b101-493c-97dc-76b99c9d942b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:20.751950 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494cb761-b101-493c-97dc-76b99c9d942b None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:20.752572 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494cb761-b101-493c-97dc-76b99c9d942b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:20.752868 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494cb761-b101-493c-97dc-76b99c9d942b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:20.754254 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494cb761-b101-493c-97dc-76b99c9d942b None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:20.765946 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 562/1103] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:55:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:20.771470 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04733787-ceb0-4c12-992c-a1c133e7f8b1 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:20.867898 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04733787-ceb0-4c12-992c-a1c133e7f8b1 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:20.900418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d018a-6edd-489c-946b-b9be5726011a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:20.901040 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d018a-6edd-489c-946b-b9be5726011a None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:20.901316 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d018a-6edd-489c-946b-b9be5726011a None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:20.901583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d018a-6edd-489c-946b-b9be5726011a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:20.907972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d018a-6edd-489c-946b-b9be5726011a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:20.909403 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d018a-6edd-489c-946b-b9be5726011a None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:20.920287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-5b0d018a-6edd-489c-946b-b9be5726011a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:55:20.920287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:20.920287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:20.920287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:20.920287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:20.920287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:20.920287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:20.920287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:20.920287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:20.920287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:20.920287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:20.920287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:20.920287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:55:20.920287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:55:20.920287 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:55:20.921826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:55:20.921826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:20.921826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:20.921826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:20.921826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:20.921826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:20.921826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:20.921826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:20.921826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:20.921826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:20.921826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:20.921826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:20.921826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:20.921826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:20.921826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:20.921826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:20.921826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:20.922823 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:55:20.922823 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:55:20.922823 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:55:20.922823 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:55:20.922823 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:55:20.922823 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:55:20.922823 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:55:20.922823 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:20.922823 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 542/1104] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:55:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:55:20.927695 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494cb761-b101-493c-97dc-76b99c9d942b None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:21.024938 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494cb761-b101-493c-97dc-76b99c9d942b None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:21.057160 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0cc05-6d37-4241-9a10-4c4e6b4fe0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:21.057756 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0cc05-6d37-4241-9a10-4c4e6b4fe0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:21.058190 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0cc05-6d37-4241-9a10-4c4e6b4fe0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:21.058478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0cc05-6d37-4241-9a10-4c4e6b4fe0c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:21.059158 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0cc05-6d37-4241-9a10-4c4e6b4fe0c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:21.059461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0cc05-6d37-4241-9a10-4c4e6b4fe0c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:21.060879 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0cc05-6d37-4241-9a10-4c4e6b4fe0c7 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:21.072851 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 563/1105] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:55:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:21.078905 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d018a-6edd-489c-946b-b9be5726011a None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:21.112980 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:21.174817 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d018a-6edd-489c-946b-b9be5726011a None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:21.206542 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c0c0e-9fde-434b-8aba-a85a381cde72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:21.207126 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c0c0e-9fde-434b-8aba-a85a381cde72 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:21.207397 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c0c0e-9fde-434b-8aba-a85a381cde72 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:21.207662 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c0c0e-9fde-434b-8aba-a85a381cde72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:21.208397 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8c0c0e-9fde-434b-8aba-a85a381cde72 None admin] RBAC: Adding query filter params (role.id=4f2c7f12acdc45b48a2271b06f69a600, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=f721a3a4eca941989685448a87bb9a19) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:21.208696 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8c0c0e-9fde-434b-8aba-a85a381cde72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:21.210124 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8c0c0e-9fde-434b-8aba-a85a381cde72 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:21.215793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 543/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:55:21 2022] GET /identity/v3/role_assignments?user.id=f721a3a4eca941989685448a87bb9a19&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=4f2c7f12acdc45b48a2271b06f69a600 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:21.890717 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6bc18b-e698-4935-bcff-f839b459e520 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:21.891267 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6bc18b-e698-4935-bcff-f839b459e520 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:21.891267 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6bc18b-e698-4935-bcff-f839b459e520 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:21.892587 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 564/1107] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:55:21.898008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb34f85-6b4c-45c9-9fc3-38e8d20c9e2f None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:21.898446 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb34f85-6b4c-45c9-9fc3-38e8d20c9e2f None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:21.898553 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb34f85-6b4c-45c9-9fc3-38e8d20c9e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:21.917958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-1cb34f85-6b4c-45c9-9fc3-38e8d20c9e2f None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:21.919189 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-1cb34f85-6b4c-45c9-9fc3-38e8d20c9e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:21.987530 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 544/1108] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:55:22.254954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6bc18b-e698-4935-bcff-f839b459e520 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:22.266948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:22.353070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6bc18b-e698-4935-bcff-f839b459e520 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:22.385004 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36bfb8e-85b0-421b-a3d2-85687ea58159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:22.385619 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36bfb8e-85b0-421b-a3d2-85687ea58159 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:22.385896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36bfb8e-85b0-421b-a3d2-85687ea58159 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:22.386193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36bfb8e-85b0-421b-a3d2-85687ea58159 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:22.387274 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36bfb8e-85b0-421b-a3d2-85687ea58159 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:22.388696 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36bfb8e-85b0-421b-a3d2-85687ea58159 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:22.391036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 565/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:55:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:22.396441 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb34f85-6b4c-45c9-9fc3-38e8d20c9e2f None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:22.425451 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:22.492077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb34f85-6b4c-45c9-9fc3-38e8d20c9e2f None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:22.524974 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd56885-55ae-4129-83f4-ee2bd430cadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:22.525511 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd56885-55ae-4129-83f4-ee2bd430cadf None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:22.525786 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd56885-55ae-4129-83f4-ee2bd430cadf None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:22.526081 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd56885-55ae-4129-83f4-ee2bd430cadf None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:22.526731 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd56885-55ae-4129-83f4-ee2bd430cadf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:22.528150 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd56885-55ae-4129-83f4-ee2bd430cadf None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:22.549333 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 545/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 14 01:55:22 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:55:24.743363 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f3f8e-1570-4aac-9e46-db16c51ab0f8 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:24.743862 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f3f8e-1570-4aac-9e46-db16c51ab0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:24.743862 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f3f8e-1570-4aac-9e46-db16c51ab0f8 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:24.745129 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 566/1111] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:55:24.750839 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe9746-1665-4bbe-a844-a4bd94e1a928 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:24.751264 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe9746-1665-4bbe-a844-a4bd94e1a928 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:24.751384 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe9746-1665-4bbe-a844-a4bd94e1a928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:24.769577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-b1fe9746-1665-4bbe-a844-a4bd94e1a928 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:24.770751 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-b1fe9746-1665-4bbe-a844-a4bd94e1a928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:24.833766 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 546/1112] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:55:25.108492 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f3f8e-1570-4aac-9e46-db16c51ab0f8 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:25.185880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:25.208667 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f3f8e-1570-4aac-9e46-db16c51ab0f8 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:25.241643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba5bdd-0280-425d-b319-00e4545c916e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:25.242285 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba5bdd-0280-425d-b319-00e4545c916e None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:25.242559 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba5bdd-0280-425d-b319-00e4545c916e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:25.242823 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba5bdd-0280-425d-b319-00e4545c916e None admin] PATH_INFO: `/v3/roles/e842ac50986946a4971a51a06a39e16b` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:25.244173 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba5bdd-0280-425d-b319-00e4545c916e None admin] RBAC: Authorizing `identity:get_role(role_id=e842ac50986946a4971a51a06a39e16b)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:25.245635 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba5bdd-0280-425d-b319-00e4545c916e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:25.246942 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 567/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:55:25 2022] GET /identity/v3/roles/e842ac50986946a4971a51a06a39e16b => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:25.253205 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe9746-1665-4bbe-a844-a4bd94e1a928 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:25.270975 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:25.349659 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe9746-1665-4bbe-a844-a4bd94e1a928 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:25.382540 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb9d25-6770-4b8a-86db-52ac2fe8f815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:25.383105 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb9d25-6770-4b8a-86db-52ac2fe8f815 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:25.383385 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb9d25-6770-4b8a-86db-52ac2fe8f815 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:25.383656 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb9d25-6770-4b8a-86db-52ac2fe8f815 None admin] PATH_INFO: `/v3/users/2a58c41f270944e2946a720273f6b075` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:25.398182 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb9d25-6770-4b8a-86db-52ac2fe8f815 None admin] RBAC: Authorizing `identity:get_user(user_id=2a58c41f270944e2946a720273f6b075)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:25.399615 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb9d25-6770-4b8a-86db-52ac2fe8f815 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:25.401225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 547/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:55:25 2022] GET /identity/v3/users/2a58c41f270944e2946a720273f6b075 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:25.407470 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba5bdd-0280-425d-b319-00e4545c916e None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:25.509023 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba5bdd-0280-425d-b319-00e4545c916e None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:25.541948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d89609-6fbc-42cf-be09-3222a1ce199e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:25.542569 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d89609-6fbc-42cf-be09-3222a1ce199e None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:25.542848 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d89609-6fbc-42cf-be09-3222a1ce199e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:25.543116 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d89609-6fbc-42cf-be09-3222a1ce199e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:25.557542 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:25.608141 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d89609-6fbc-42cf-be09-3222a1ce199e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:25.609611 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d89609-6fbc-42cf-be09-3222a1ce199e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:25.703339 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 568/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:55:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:55:25.709199 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb9d25-6770-4b8a-86db-52ac2fe8f815 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:25.780493 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:25.807432 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb9d25-6770-4b8a-86db-52ac2fe8f815 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:25.840438 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b390de0-e78a-4930-883e-18ee3f34b933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:25.841048 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b390de0-e78a-4930-883e-18ee3f34b933 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:25.841325 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b390de0-e78a-4930-883e-18ee3f34b933 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:25.841591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b390de0-e78a-4930-883e-18ee3f34b933 None admin] PATH_INFO: `/v3/projects/583f9ca7c28e4637bf9266badb4c6b56` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:25.842749 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b390de0-e78a-4930-883e-18ee3f34b933 None admin] RBAC: Authorizing `identity:get_project(project_id=583f9ca7c28e4637bf9266badb4c6b56)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:25.844174 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b390de0-e78a-4930-883e-18ee3f34b933 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:25.845802 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 548/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:55:25 2022] GET /identity/v3/projects/583f9ca7c28e4637bf9266badb4c6b56 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:25.852087 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d89609-6fbc-42cf-be09-3222a1ce199e None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:25.898092 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:25.948136 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d89609-6fbc-42cf-be09-3222a1ce199e None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:25.983072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e627c6-6ab1-4293-a441-c97ed2da9d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:25.983643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e627c6-6ab1-4293-a441-c97ed2da9d1b None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:25.983921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e627c6-6ab1-4293-a441-c97ed2da9d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:25.984191 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e627c6-6ab1-4293-a441-c97ed2da9d1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:25.984898 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e627c6-6ab1-4293-a441-c97ed2da9d1b None admin] RBAC: Adding query filter params (role.id=e842ac50986946a4971a51a06a39e16b, scope.project.id=583f9ca7c28e4637bf9266badb4c6b56, user.id=2a58c41f270944e2946a720273f6b075) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:25.985202 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e627c6-6ab1-4293-a441-c97ed2da9d1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:25.986651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e627c6-6ab1-4293-a441-c97ed2da9d1b None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:25.991945 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 569/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:55:25 2022] GET /identity/v3/role_assignments?user.id=2a58c41f270944e2946a720273f6b075&scope.project.id=583f9ca7c28e4637bf9266badb4c6b56&role.id=e842ac50986946a4971a51a06a39e16b => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:28.113155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035e6d0-94f3-4d08-a963-935e6a4d9899 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:28.113691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035e6d0-94f3-4d08-a963-935e6a4d9899 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:28.113691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035e6d0-94f3-4d08-a963-935e6a4d9899 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:28.115075 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 549/1118] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:55:28.120196 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf96316-ca2c-48d3-b6b1-3f4b7ef55f66 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:28.120578 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf96316-ca2c-48d3-b6b1-3f4b7ef55f66 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:28.120739 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf96316-ca2c-48d3-b6b1-3f4b7ef55f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:28.139503 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-bdf96316-ca2c-48d3-b6b1-3f4b7ef55f66 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:28.141185 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-bdf96316-ca2c-48d3-b6b1-3f4b7ef55f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:28.207204 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 570/1119] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:55:28.478214 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035e6d0-94f3-4d08-a963-935e6a4d9899 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:28.574603 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035e6d0-94f3-4d08-a963-935e6a4d9899 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:28.607250 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a77a96-936b-4fbd-b8c6-a87a95a1bf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:28.607858 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a77a96-936b-4fbd-b8c6-a87a95a1bf71 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:28.608186 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a77a96-936b-4fbd-b8c6-a87a95a1bf71 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:28.608459 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a77a96-936b-4fbd-b8c6-a87a95a1bf71 None admin] PATH_INFO: `/v3/roles/e842ac50986946a4971a51a06a39e16b` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:28.609900 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a77a96-936b-4fbd-b8c6-a87a95a1bf71 None admin] RBAC: Authorizing `identity:get_role(role_id=e842ac50986946a4971a51a06a39e16b)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:28.611294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a77a96-936b-4fbd-b8c6-a87a95a1bf71 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:28.612596 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 550/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:55:28 2022] GET /identity/v3/roles/e842ac50986946a4971a51a06a39e16b => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:55:28.618572 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf96316-ca2c-48d3-b6b1-3f4b7ef55f66 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:28.658426 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:28.713151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf96316-ca2c-48d3-b6b1-3f4b7ef55f66 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:28.745038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa675b52-3978-462e-8b13-971cce1aa4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:28.745648 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa675b52-3978-462e-8b13-971cce1aa4ce None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:28.745922 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa675b52-3978-462e-8b13-971cce1aa4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:28.746218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa675b52-3978-462e-8b13-971cce1aa4ce None admin] PATH_INFO: `/v3/users/2a58c41f270944e2946a720273f6b075` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:28.747449 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa675b52-3978-462e-8b13-971cce1aa4ce None admin] RBAC: Authorizing `identity:get_user(user_id=2a58c41f270944e2946a720273f6b075)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:28.749424 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa675b52-3978-462e-8b13-971cce1aa4ce None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:28.750992 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 571/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:55:28 2022] GET /identity/v3/users/2a58c41f270944e2946a720273f6b075 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:28.757675 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a77a96-936b-4fbd-b8c6-a87a95a1bf71 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:28.771430 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:28.854463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a77a96-936b-4fbd-b8c6-a87a95a1bf71 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:28.886828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2208bfbc-1583-435b-a770-a48f236e60bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:28.887454 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208bfbc-1583-435b-a770-a48f236e60bc None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:28.887731 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208bfbc-1583-435b-a770-a48f236e60bc None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:28.887999 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208bfbc-1583-435b-a770-a48f236e60bc None admin] PATH_INFO: `/v3/projects/583f9ca7c28e4637bf9266badb4c6b56` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:28.889094 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2208bfbc-1583-435b-a770-a48f236e60bc None admin] RBAC: Authorizing `identity:get_project(project_id=583f9ca7c28e4637bf9266badb4c6b56)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:28.891122 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2208bfbc-1583-435b-a770-a48f236e60bc None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:28.892716 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 551/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:55:28 2022] GET /identity/v3/projects/583f9ca7c28e4637bf9266badb4c6b56 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:28.898618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa675b52-3978-462e-8b13-971cce1aa4ce None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:28.992255 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa675b52-3978-462e-8b13-971cce1aa4ce None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:29.001400 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:29.025977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12bdeea-0329-4448-9441-7ac02b16572b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:29.026640 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bdeea-0329-4448-9441-7ac02b16572b None admin] REQUEST_METHOD: `PUT` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:29.026915 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bdeea-0329-4448-9441-7ac02b16572b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:29.027181 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bdeea-0329-4448-9441-7ac02b16572b None admin] PATH_INFO: `/v3/projects/583f9ca7c28e4637bf9266badb4c6b56/users/2a58c41f270944e2946a720273f6b075/roles/e842ac50986946a4971a51a06a39e16b` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:29.029261 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12bdeea-0329-4448-9441-7ac02b16572b None admin] RBAC: Authorizing `identity:create_grant(project_id=583f9ca7c28e4637bf9266badb4c6b56, user_id=2a58c41f270944e2946a720273f6b075, role_id=e842ac50986946a4971a51a06a39e16b)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:29.030783 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12bdeea-0329-4448-9441-7ac02b16572b None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:29.042494 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 572/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:55:28 2022] PUT /identity/v3/projects/583f9ca7c28e4637bf9266badb4c6b56/users/2a58c41f270944e2946a720273f6b075/roles/e842ac50986946a4971a51a06a39e16b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:55:31.153782 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb5228-0f0c-426a-8f4b-4e684e24d548 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:31.154345 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb5228-0f0c-426a-8f4b-4e684e24d548 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:31.154345 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb5228-0f0c-426a-8f4b-4e684e24d548 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:31.155733 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 552/1124] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:55:31.160992 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb36dd7-679e-4d80-80c2-5218ae35fe53 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:31.161366 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb36dd7-679e-4d80-80c2-5218ae35fe53 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:31.161533 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb36dd7-679e-4d80-80c2-5218ae35fe53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:31.179104 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-feb36dd7-679e-4d80-80c2-5218ae35fe53 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:31.180261 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-feb36dd7-679e-4d80-80c2-5218ae35fe53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:31.242571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 573/1125] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:55:31.511930 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb5228-0f0c-426a-8f4b-4e684e24d548 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:31.587676 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:31.609337 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb5228-0f0c-426a-8f4b-4e684e24d548 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:31.641679 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ab7c9-9d1b-4d47-9fff-60daf6b3d876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:31.642313 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ab7c9-9d1b-4d47-9fff-60daf6b3d876 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:31.642590 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ab7c9-9d1b-4d47-9fff-60daf6b3d876 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:31.642856 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ab7c9-9d1b-4d47-9fff-60daf6b3d876 None admin] PATH_INFO: `/v3/roles/e842ac50986946a4971a51a06a39e16b` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:31.644226 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ab7c9-9d1b-4d47-9fff-60daf6b3d876 None admin] RBAC: Authorizing `identity:get_role(role_id=e842ac50986946a4971a51a06a39e16b)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:31.645573 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ab7c9-9d1b-4d47-9fff-60daf6b3d876 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:31.646887 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 553/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:55:31 2022] GET /identity/v3/roles/e842ac50986946a4971a51a06a39e16b => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:31.652524 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb36dd7-679e-4d80-80c2-5218ae35fe53 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:31.743387 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:31.747324 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb36dd7-679e-4d80-80c2-5218ae35fe53 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:31.781678 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8be1f8-2c42-4ff4-ab29-4209e25dded1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:31.782269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8be1f8-2c42-4ff4-ab29-4209e25dded1 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:31.782549 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8be1f8-2c42-4ff4-ab29-4209e25dded1 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:31.782820 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8be1f8-2c42-4ff4-ab29-4209e25dded1 None admin] PATH_INFO: `/v3/users/2a58c41f270944e2946a720273f6b075` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:31.783925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8be1f8-2c42-4ff4-ab29-4209e25dded1 None admin] RBAC: Authorizing `identity:get_user(user_id=2a58c41f270944e2946a720273f6b075)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:31.785348 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8be1f8-2c42-4ff4-ab29-4209e25dded1 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:31.786928 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 574/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:55:31 2022] GET /identity/v3/users/2a58c41f270944e2946a720273f6b075 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:31.792583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ab7c9-9d1b-4d47-9fff-60daf6b3d876 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:31.887334 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ab7c9-9d1b-4d47-9fff-60daf6b3d876 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:31.919534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de09522-c0bb-499a-8083-923dce5c359d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:31.920116 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de09522-c0bb-499a-8083-923dce5c359d None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:31.920415 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de09522-c0bb-499a-8083-923dce5c359d None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:31.920700 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de09522-c0bb-499a-8083-923dce5c359d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:31.934898 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:31.985728 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de09522-c0bb-499a-8083-923dce5c359d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:31.987191 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de09522-c0bb-499a-8083-923dce5c359d None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:32.084788 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 554/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:55:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:55:32.090570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8be1f8-2c42-4ff4-ab29-4209e25dded1 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:32.193856 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8be1f8-2c42-4ff4-ab29-4209e25dded1 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:32.225768 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd19ecc-314d-49ab-9756-04b16f512d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:32.226416 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd19ecc-314d-49ab-9756-04b16f512d39 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:32.226695 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd19ecc-314d-49ab-9756-04b16f512d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:32.226967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd19ecc-314d-49ab-9756-04b16f512d39 None admin] PATH_INFO: `/v3/projects/583f9ca7c28e4637bf9266badb4c6b56` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:32.228060 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd19ecc-314d-49ab-9756-04b16f512d39 None admin] RBAC: Authorizing `identity:get_project(project_id=583f9ca7c28e4637bf9266badb4c6b56)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:32.229481 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd19ecc-314d-49ab-9756-04b16f512d39 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:32.231111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 575/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:55:32 2022] GET /identity/v3/projects/583f9ca7c28e4637bf9266badb4c6b56 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:32.237807 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de09522-c0bb-499a-8083-923dce5c359d None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:32.282525 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:32.334448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de09522-c0bb-499a-8083-923dce5c359d None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:32.366886 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a5d3b-2525-41f8-b589-e8df00aaa424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:32.367475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a5d3b-2525-41f8-b589-e8df00aaa424 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:32.367750 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a5d3b-2525-41f8-b589-e8df00aaa424 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:32.368019 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a5d3b-2525-41f8-b589-e8df00aaa424 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:32.368726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a5d3b-2525-41f8-b589-e8df00aaa424 None admin] RBAC: Adding query filter params (role.id=e842ac50986946a4971a51a06a39e16b, scope.project.id=583f9ca7c28e4637bf9266badb4c6b56, user.id=2a58c41f270944e2946a720273f6b075) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:32.369034 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a5d3b-2525-41f8-b589-e8df00aaa424 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:32.370476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a5d3b-2525-41f8-b589-e8df00aaa424 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:32.376581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 555/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:55:32 2022] GET /identity/v3/role_assignments?user.id=2a58c41f270944e2946a720273f6b075&scope.project.id=583f9ca7c28e4637bf9266badb4c6b56&role.id=e842ac50986946a4971a51a06a39e16b => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:34.495964 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b8512-a4d2-41dc-9773-8e72ab029f01 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:34.496512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b8512-a4d2-41dc-9773-8e72ab029f01 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:34.496512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b8512-a4d2-41dc-9773-8e72ab029f01 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:34.497814 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 576/1131] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:55:34.503140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53d59c-4b55-46d7-b8f9-5cdb8a9ffa2a None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:34.503538 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53d59c-4b55-46d7-b8f9-5cdb8a9ffa2a None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:34.503684 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53d59c-4b55-46d7-b8f9-5cdb8a9ffa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:34.522135 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-7b53d59c-4b55-46d7-b8f9-5cdb8a9ffa2a None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:34.523306 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-7b53d59c-4b55-46d7-b8f9-5cdb8a9ffa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:34.587019 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 556/1132] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:55:34.855893 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b8512-a4d2-41dc-9773-8e72ab029f01 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:34.890453 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:34.953736 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b8512-a4d2-41dc-9773-8e72ab029f01 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:34.986088 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e34ad-c2d0-4601-ae59-4d5f5beecdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:34.986692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e34ad-c2d0-4601-ae59-4d5f5beecdf5 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:34.986965 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e34ad-c2d0-4601-ae59-4d5f5beecdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:34.987233 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e34ad-c2d0-4601-ae59-4d5f5beecdf5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:34.988307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e34ad-c2d0-4601-ae59-4d5f5beecdf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:34.989711 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e34ad-c2d0-4601-ae59-4d5f5beecdf5 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:34.991231 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 577/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:55:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:34.996895 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53d59c-4b55-46d7-b8f9-5cdb8a9ffa2a None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:35.037623 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:35.093388 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53d59c-4b55-46d7-b8f9-5cdb8a9ffa2a None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:35.126046 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4ee4a-80bb-4448-960e-29e2c2cb64fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:35.126582 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4ee4a-80bb-4448-960e-29e2c2cb64fd None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:35.126859 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4ee4a-80bb-4448-960e-29e2c2cb64fd None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:35.127128 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4ee4a-80bb-4448-960e-29e2c2cb64fd None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:35.127764 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4ee4a-80bb-4448-960e-29e2c2cb64fd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:35.129173 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4ee4a-80bb-4448-960e-29e2c2cb64fd None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:35.154285 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 557/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 14 01:55:34 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:55:37.289751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab329e-2968-48d7-9dd0-156074b6b352 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:37.290318 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab329e-2968-48d7-9dd0-156074b6b352 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:37.290318 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab329e-2968-48d7-9dd0-156074b6b352 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:37.291619 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 578/1135] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:55:37.297317 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767cb5ac-187c-41c3-a9cb-3b16718820bf None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:37.297707 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767cb5ac-187c-41c3-a9cb-3b16718820bf None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:37.297864 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767cb5ac-187c-41c3-a9cb-3b16718820bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:37.315635 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-767cb5ac-187c-41c3-a9cb-3b16718820bf None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:37.316805 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-767cb5ac-187c-41c3-a9cb-3b16718820bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:37.381835 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 558/1136] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:55:37.651462 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab329e-2968-48d7-9dd0-156074b6b352 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:37.748287 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab329e-2968-48d7-9dd0-156074b6b352 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:37.750495 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:37.783166 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37ce78-25f1-4e7e-9465-1ed87bef2e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:37.783785 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37ce78-25f1-4e7e-9465-1ed87bef2e27 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:37.784062 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37ce78-25f1-4e7e-9465-1ed87bef2e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:37.784330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37ce78-25f1-4e7e-9465-1ed87bef2e27 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:37.785420 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37ce78-25f1-4e7e-9465-1ed87bef2e27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:37.786873 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37ce78-25f1-4e7e-9465-1ed87bef2e27 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:37.788393 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 579/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:55:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:37.794180 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767cb5ac-187c-41c3-a9cb-3b16718820bf None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:37.817437 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:37.891918 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767cb5ac-187c-41c3-a9cb-3b16718820bf None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:37.927015 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0707a1d-f82f-4879-87eb-4e1223eb86d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:37.927550 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0707a1d-f82f-4879-87eb-4e1223eb86d9 None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:37.927850 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0707a1d-f82f-4879-87eb-4e1223eb86d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:37.928104 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0707a1d-f82f-4879-87eb-4e1223eb86d9 None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:37.928746 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0707a1d-f82f-4879-87eb-4e1223eb86d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:37.930198 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0707a1d-f82f-4879-87eb-4e1223eb86d9 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:37.950710 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 559/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 14 01:55:37 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:55:40.103976 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55384809-02c8-4d61-bd47-617c5bc26c49 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:40.104547 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55384809-02c8-4d61-bd47-617c5bc26c49 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:40.104547 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55384809-02c8-4d61-bd47-617c5bc26c49 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:40.105806 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 580/1139] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:55:40.111059 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3de95-b64a-475c-b777-f12d82c261aa None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:40.111430 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3de95-b64a-475c-b777-f12d82c261aa None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:40.111595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3de95-b64a-475c-b777-f12d82c261aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:40.130790 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-83b3de95-b64a-475c-b777-f12d82c261aa None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:40.131945 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-83b3de95-b64a-475c-b777-f12d82c261aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:40.195414 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 560/1140] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:55:40.465498 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55384809-02c8-4d61-bd47-617c5bc26c49 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:40.564010 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55384809-02c8-4d61-bd47-617c5bc26c49 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:40.596716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1aafc-7ed6-4425-990b-0dbc54dd494a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:40.597332 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1aafc-7ed6-4425-990b-0dbc54dd494a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:40.597609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1aafc-7ed6-4425-990b-0dbc54dd494a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:40.597880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1aafc-7ed6-4425-990b-0dbc54dd494a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:40.607561 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1aafc-7ed6-4425-990b-0dbc54dd494a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:55:40.612330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-e6a1aafc-7ed6-4425-990b-0dbc54dd494a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:55:40.612330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:40.612330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:40.612330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:40.612330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:40.612330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:40.612330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:40.612330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:40.612330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:40.612330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:40.612330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:40.612330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:40.612330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:55:40.612330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:55:40.612330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:55:40.612330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:55:40.613448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:55:40.613448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:55:40.613448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:55:40.613448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:55:40.613448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:40.613448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:40.613448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:40.613448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:40.613448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:40.613448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:40.613448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:40.613448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:40.613448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:40.613448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:40.613448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:40.613448 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:40.614641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:40.614641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:40.614641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:40.614641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:40.614641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:55:40.614641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:55:40.614641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:55:40.614641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:55:40.614641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:55:40.614641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:55:40.614641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:55:40.614641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:55:40.614641 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 581/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:55:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:55:40.619627 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3de95-b64a-475c-b777-f12d82c261aa None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:40.629741 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:40.718208 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3de95-b64a-475c-b777-f12d82c261aa None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:40.750145 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f11842-ff4d-4727-a4c4-0359204794cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:40.750735 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f11842-ff4d-4727-a4c4-0359204794cc None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:40.751012 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f11842-ff4d-4727-a4c4-0359204794cc None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:40.751276 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f11842-ff4d-4727-a4c4-0359204794cc None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:40.751917 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f11842-ff4d-4727-a4c4-0359204794cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:40.752211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f11842-ff4d-4727-a4c4-0359204794cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:40.753555 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f11842-ff4d-4727-a4c4-0359204794cc None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:40.761561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 561/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:55:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:40.767506 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1aafc-7ed6-4425-990b-0dbc54dd494a None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:40.798327 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:40.867716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1aafc-7ed6-4425-990b-0dbc54dd494a None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:40.899907 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef233b94-d5c3-4d41-93e7-dd589ac41421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:40.900525 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef233b94-d5c3-4d41-93e7-dd589ac41421 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:40.900801 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef233b94-d5c3-4d41-93e7-dd589ac41421 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:40.901070 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef233b94-d5c3-4d41-93e7-dd589ac41421 None admin] PATH_INFO: `/v3/users/e2ec921e86c94369b732429cf7ba935a` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:40.914075 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef233b94-d5c3-4d41-93e7-dd589ac41421 None admin] RBAC: Authorizing `identity:get_user(user_id=e2ec921e86c94369b732429cf7ba935a)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:40.915519 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef233b94-d5c3-4d41-93e7-dd589ac41421 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:40.917111 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 582/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:55:40 2022] GET /identity/v3/users/e2ec921e86c94369b732429cf7ba935a => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:40.923644 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f11842-ff4d-4727-a4c4-0359204794cc None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:40.989493 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:41.019247 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f11842-ff4d-4727-a4c4-0359204794cc None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:41.051775 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e115ae7-661c-4469-80c0-936f05477ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:41.052359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e115ae7-661c-4469-80c0-936f05477ada None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:41.052634 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e115ae7-661c-4469-80c0-936f05477ada None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:41.052902 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e115ae7-661c-4469-80c0-936f05477ada None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:41.116449 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e115ae7-661c-4469-80c0-936f05477ada None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:41.117875 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e115ae7-661c-4469-80c0-936f05477ada None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:41.182123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:41.218382 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 562/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:55:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:55:41.224418 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef233b94-d5c3-4d41-93e7-dd589ac41421 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:41.308328 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:41.319824 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef233b94-d5c3-4d41-93e7-dd589ac41421 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:41.352682 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31922ff1-4ee4-43c9-ac56-cdfbeb9f7618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:41.353293 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31922ff1-4ee4-43c9-ac56-cdfbeb9f7618 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:41.353567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31922ff1-4ee4-43c9-ac56-cdfbeb9f7618 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:41.353835 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31922ff1-4ee4-43c9-ac56-cdfbeb9f7618 None admin] PATH_INFO: `/v3/projects/2ce21ff39736428fa2e883677e75f8ed` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:41.354958 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31922ff1-4ee4-43c9-ac56-cdfbeb9f7618 None admin] RBAC: Authorizing `identity:get_project(project_id=2ce21ff39736428fa2e883677e75f8ed)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:41.356382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31922ff1-4ee4-43c9-ac56-cdfbeb9f7618 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:41.358000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 583/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:55:41 2022] GET /identity/v3/projects/2ce21ff39736428fa2e883677e75f8ed => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:41.364939 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e115ae7-661c-4469-80c0-936f05477ada None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:41.461206 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e115ae7-661c-4469-80c0-936f05477ada None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:41.494751 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c408ee5-b962-4b42-a4fb-8bfdb47ea083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:41.495350 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c408ee5-b962-4b42-a4fb-8bfdb47ea083 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:41.495628 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c408ee5-b962-4b42-a4fb-8bfdb47ea083 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:41.495907 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c408ee5-b962-4b42-a4fb-8bfdb47ea083 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:41.496622 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c408ee5-b962-4b42-a4fb-8bfdb47ea083 None admin] RBAC: Adding query filter params (role.id=aa420c2fb1ab4bb3b05b004a4d89779a, scope.project.id=2ce21ff39736428fa2e883677e75f8ed, user.id=e2ec921e86c94369b732429cf7ba935a) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:41.496927 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c408ee5-b962-4b42-a4fb-8bfdb47ea083 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:41.498498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c408ee5-b962-4b42-a4fb-8bfdb47ea083 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:41.505051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 563/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:55:41 2022] GET /identity/v3/role_assignments?user.id=e2ec921e86c94369b732429cf7ba935a&scope.project.id=2ce21ff39736428fa2e883677e75f8ed&role.id=aa420c2fb1ab4bb3b05b004a4d89779a => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:43.595363 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070a0ac-c0b2-41b8-8956-5f721e95c9a8 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:43.595923 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070a0ac-c0b2-41b8-8956-5f721e95c9a8 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:43.595923 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070a0ac-c0b2-41b8-8956-5f721e95c9a8 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:43.597212 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 584/1147] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:55:43.602585 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ff0df-a569-42d6-8570-3ae9da631750 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:43.602983 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ff0df-a569-42d6-8570-3ae9da631750 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:43.603137 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ff0df-a569-42d6-8570-3ae9da631750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:43.620863 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-754ff0df-a569-42d6-8570-3ae9da631750 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:43.622072 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-754ff0df-a569-42d6-8570-3ae9da631750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:43.648123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:43.687650 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 564/1148] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:55:43.963280 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070a0ac-c0b2-41b8-8956-5f721e95c9a8 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:44.060049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070a0ac-c0b2-41b8-8956-5f721e95c9a8 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:44.092845 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91c7db-9f2c-4246-bafd-498d4d27bd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:44.093456 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91c7db-9f2c-4246-bafd-498d4d27bd4b None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:44.093734 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91c7db-9f2c-4246-bafd-498d4d27bd4b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:44.094022 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91c7db-9f2c-4246-bafd-498d4d27bd4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:44.103619 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91c7db-9f2c-4246-bafd-498d4d27bd4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:55:44.108866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-6d91c7db-9f2c-4246-bafd-498d4d27bd4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:55:44.108866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:44.108866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:44.108866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:44.108866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:44.108866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:44.108866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:44.108866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:44.108866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:44.108866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:44.108866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:44.108866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:44.108866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:55:44.108866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:55:44.108866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:55:44.108866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:55:44.110011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:55:44.110011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:55:44.110011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:55:44.110011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:55:44.110011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:44.110011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:44.110011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:44.110011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:44.110011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:44.110011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:44.110011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:44.110011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:44.110011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:44.110011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:44.110011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:44.110011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:44.110927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:44.110927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:44.110927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:44.110927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:44.110927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:55:44.110927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:55:44.110927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:55:44.110927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:55:44.110927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:55:44.110927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:55:44.110927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:55:44.110927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:55:44.110927 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 585/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:55:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:55:44.115918 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ff0df-a569-42d6-8570-3ae9da631750 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:44.215448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ff0df-a569-42d6-8570-3ae9da631750 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:44.247296 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36315de-ed4e-4a5f-9c31-3b254c311aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:44.247886 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36315de-ed4e-4a5f-9c31-3b254c311aaf None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:44.248714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36315de-ed4e-4a5f-9c31-3b254c311aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:44.248979 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36315de-ed4e-4a5f-9c31-3b254c311aaf None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:44.249619 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36315de-ed4e-4a5f-9c31-3b254c311aaf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:44.249910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36315de-ed4e-4a5f-9c31-3b254c311aaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:44.251279 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36315de-ed4e-4a5f-9c31-3b254c311aaf None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:44.259313 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 565/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:55:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:44.265130 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91c7db-9f2c-4246-bafd-498d4d27bd4b None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:44.283138 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:44.362183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91c7db-9f2c-4246-bafd-498d4d27bd4b None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:44.396312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e91f8f-0cb5-45bf-8e7f-af22130b02a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:44.396937 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e91f8f-0cb5-45bf-8e7f-af22130b02a8 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:44.397217 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e91f8f-0cb5-45bf-8e7f-af22130b02a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:44.397488 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e91f8f-0cb5-45bf-8e7f-af22130b02a8 None admin] PATH_INFO: `/v3/users/e2ec921e86c94369b732429cf7ba935a` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:44.398614 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e91f8f-0cb5-45bf-8e7f-af22130b02a8 None admin] RBAC: Authorizing `identity:get_user(user_id=e2ec921e86c94369b732429cf7ba935a)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:44.400041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e91f8f-0cb5-45bf-8e7f-af22130b02a8 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:44.401583 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 586/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:55:44 2022] GET /identity/v3/users/e2ec921e86c94369b732429cf7ba935a => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:44.407907 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36315de-ed4e-4a5f-9c31-3b254c311aaf None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:44.434737 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:44.504705 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36315de-ed4e-4a5f-9c31-3b254c311aaf None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:44.536841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff18a2-3b1a-4bf3-acab-ff2ef96aa3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:44.537454 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff18a2-3b1a-4bf3-acab-ff2ef96aa3cc None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:44.537732 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff18a2-3b1a-4bf3-acab-ff2ef96aa3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:44.538019 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff18a2-3b1a-4bf3-acab-ff2ef96aa3cc None admin] PATH_INFO: `/v3/projects/2ce21ff39736428fa2e883677e75f8ed` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:44.539118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff18a2-3b1a-4bf3-acab-ff2ef96aa3cc None admin] RBAC: Authorizing `identity:get_project(project_id=2ce21ff39736428fa2e883677e75f8ed)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:44.540541 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff18a2-3b1a-4bf3-acab-ff2ef96aa3cc None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:44.542172 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 566/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:55:44 2022] GET /identity/v3/projects/2ce21ff39736428fa2e883677e75f8ed => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:44.547890 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e91f8f-0cb5-45bf-8e7f-af22130b02a8 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:44.627736 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:44.643816 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e91f8f-0cb5-45bf-8e7f-af22130b02a8 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:44.675321 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894405e-5484-47a4-b01a-7aae7214b3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:44.675954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894405e-5484-47a4-b01a-7aae7214b3fa None admin] REQUEST_METHOD: `PUT` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:44.676227 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894405e-5484-47a4-b01a-7aae7214b3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:44.676493 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894405e-5484-47a4-b01a-7aae7214b3fa None admin] PATH_INFO: `/v3/projects/2ce21ff39736428fa2e883677e75f8ed/users/e2ec921e86c94369b732429cf7ba935a/roles/aa420c2fb1ab4bb3b05b004a4d89779a` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:44.678371 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8894405e-5484-47a4-b01a-7aae7214b3fa None admin] RBAC: Authorizing `identity:create_grant(project_id=2ce21ff39736428fa2e883677e75f8ed, user_id=e2ec921e86c94369b732429cf7ba935a, role_id=aa420c2fb1ab4bb3b05b004a4d89779a)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:44.679784 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8894405e-5484-47a4-b01a-7aae7214b3fa None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:44.692746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 587/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:55:44 2022] PUT /identity/v3/projects/2ce21ff39736428fa2e883677e75f8ed/users/e2ec921e86c94369b732429cf7ba935a/roles/aa420c2fb1ab4bb3b05b004a4d89779a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:55:46.792856 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cd712-ad78-4557-9eec-a2aa0ae1caeb None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:46.793403 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cd712-ad78-4557-9eec-a2aa0ae1caeb None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:46.793403 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cd712-ad78-4557-9eec-a2aa0ae1caeb None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:46.794667 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 567/1154] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:55:46.800276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858d078-c336-45fc-a392-1f96b3babb96 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:46.800671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858d078-c336-45fc-a392-1f96b3babb96 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:46.800809 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858d078-c336-45fc-a392-1f96b3babb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:46.818411 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-5858d078-c336-45fc-a392-1f96b3babb96 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:46.819576 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-5858d078-c336-45fc-a392-1f96b3babb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:46.882515 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 588/1155] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:55:47.151476 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cd712-ad78-4557-9eec-a2aa0ae1caeb None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:47.241610 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:47.249163 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cd712-ad78-4557-9eec-a2aa0ae1caeb None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:47.282515 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef9914-8289-43db-898f-75bca4b8f7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:47.283140 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef9914-8289-43db-898f-75bca4b8f7cb None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:47.283420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef9914-8289-43db-898f-75bca4b8f7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:47.283692 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef9914-8289-43db-898f-75bca4b8f7cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:47.294037 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef9914-8289-43db-898f-75bca4b8f7cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:55:47.298846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-1cef9914-8289-43db-898f-75bca4b8f7cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:55:47.298846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:47.298846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:47.298846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:47.298846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:47.298846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:47.298846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:47.298846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:47.298846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:47.298846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:47.298846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:47.298846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:47.298846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:55:47.298846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:55:47.298846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:55:47.298846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:55:47.300132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:55:47.300132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:55:47.300132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:55:47.300132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:55:47.300132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:47.300132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:47.300132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:47.300132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:47.300132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:47.300132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:47.300132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:47.300132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:47.300132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:47.300132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:47.300132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:47.300132 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:47.301233 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:47.301233 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:47.301233 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:47.301233 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:47.301233 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:55:47.301233 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:55:47.301233 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:55:47.301233 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:55:47.301233 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:55:47.301233 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:55:47.301233 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:55:47.301233 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:55:47.301233 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 568/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:55:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:55:47.305736 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858d078-c336-45fc-a392-1f96b3babb96 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:47.375717 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:47.401629 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858d078-c336-45fc-a392-1f96b3babb96 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:47.434920 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca3bbc-3c06-4bf0-ac59-5d1f1552333b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:47.436026 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca3bbc-3c06-4bf0-ac59-5d1f1552333b None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:47.436301 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca3bbc-3c06-4bf0-ac59-5d1f1552333b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:47.436567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca3bbc-3c06-4bf0-ac59-5d1f1552333b None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:47.437212 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca3bbc-3c06-4bf0-ac59-5d1f1552333b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:47.437509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca3bbc-3c06-4bf0-ac59-5d1f1552333b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:47.438969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca3bbc-3c06-4bf0-ac59-5d1f1552333b None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:47.446672 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 589/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:55:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:47.452529 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef9914-8289-43db-898f-75bca4b8f7cb None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:47.552604 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef9914-8289-43db-898f-75bca4b8f7cb None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:47.584105 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d4ef2-c15b-43be-9a9c-c8fb7e1ba319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:47.584726 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d4ef2-c15b-43be-9a9c-c8fb7e1ba319 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:47.585007 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d4ef2-c15b-43be-9a9c-c8fb7e1ba319 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:47.585278 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d4ef2-c15b-43be-9a9c-c8fb7e1ba319 None admin] PATH_INFO: `/v3/users/e2ec921e86c94369b732429cf7ba935a` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:47.586412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d4ef2-c15b-43be-9a9c-c8fb7e1ba319 None admin] RBAC: Authorizing `identity:get_user(user_id=e2ec921e86c94369b732429cf7ba935a)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:47.587911 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d4ef2-c15b-43be-9a9c-c8fb7e1ba319 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:47.589520 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 569/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:55:47 2022] GET /identity/v3/users/e2ec921e86c94369b732429cf7ba935a => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:47.595866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca3bbc-3c06-4bf0-ac59-5d1f1552333b None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:47.691727 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca3bbc-3c06-4bf0-ac59-5d1f1552333b None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:47.723222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4fe9f-42a4-4a1b-b118-3c5ec7f26112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:47.723811 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4fe9f-42a4-4a1b-b118-3c5ec7f26112 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:47.724087 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4fe9f-42a4-4a1b-b118-3c5ec7f26112 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:47.724355 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4fe9f-42a4-4a1b-b118-3c5ec7f26112 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:47.727566 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:47.788677 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f4fe9f-42a4-4a1b-b118-3c5ec7f26112 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:47.790142 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f4fe9f-42a4-4a1b-b118-3c5ec7f26112 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:47.883586 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 590/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:55:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:55:47.889615 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d4ef2-c15b-43be-9a9c-c8fb7e1ba319 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:47.915969 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:47.989355 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d4ef2-c15b-43be-9a9c-c8fb7e1ba319 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:48.021989 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a1f60-9499-4bd6-b938-83b721664054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:48.022626 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a1f60-9499-4bd6-b938-83b721664054 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:48.022904 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a1f60-9499-4bd6-b938-83b721664054 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:48.023178 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a1f60-9499-4bd6-b938-83b721664054 None admin] PATH_INFO: `/v3/projects/2ce21ff39736428fa2e883677e75f8ed` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:48.024283 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a1f60-9499-4bd6-b938-83b721664054 None admin] RBAC: Authorizing `identity:get_project(project_id=2ce21ff39736428fa2e883677e75f8ed)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:48.025710 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a1f60-9499-4bd6-b938-83b721664054 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:48.027338 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 570/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:55:47 2022] GET /identity/v3/projects/2ce21ff39736428fa2e883677e75f8ed => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:48.034145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4fe9f-42a4-4a1b-b118-3c5ec7f26112 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:48.068213 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:48.131227 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4fe9f-42a4-4a1b-b118-3c5ec7f26112 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:48.164076 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3d2df-e98f-476e-b67e-c4baa58e6c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:48.164670 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3d2df-e98f-476e-b67e-c4baa58e6c86 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:48.164947 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3d2df-e98f-476e-b67e-c4baa58e6c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:48.165217 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3d2df-e98f-476e-b67e-c4baa58e6c86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:48.165921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3d2df-e98f-476e-b67e-c4baa58e6c86 None admin] RBAC: Adding query filter params (role.id=aa420c2fb1ab4bb3b05b004a4d89779a, scope.project.id=2ce21ff39736428fa2e883677e75f8ed, user.id=e2ec921e86c94369b732429cf7ba935a) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:48.166253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3d2df-e98f-476e-b67e-c4baa58e6c86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:48.167666 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3d2df-e98f-476e-b67e-c4baa58e6c86 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:48.173283 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 591/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:55:48 2022] GET /identity/v3/role_assignments?user.id=e2ec921e86c94369b732429cf7ba935a&scope.project.id=2ce21ff39736428fa2e883677e75f8ed&role.id=aa420c2fb1ab4bb3b05b004a4d89779a => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:50.284231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883fadb-5772-4956-afd0-4d4a07bf8b80 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:50.284793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883fadb-5772-4956-afd0-4d4a07bf8b80 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:50.284793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883fadb-5772-4956-afd0-4d4a07bf8b80 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:50.286128 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 571/1162] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:55:50.291163 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5693d-b132-4e22-95f5-4853708be506 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:50.291574 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5693d-b132-4e22-95f5-4853708be506 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:50.291702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5693d-b132-4e22-95f5-4853708be506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:50.309367 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-46d5693d-b132-4e22-95f5-4853708be506 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:50.310618 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-46d5693d-b132-4e22-95f5-4853708be506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:50.373810 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 592/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:55:50.638990 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883fadb-5772-4956-afd0-4d4a07bf8b80 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:50.728177 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:50.739126 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883fadb-5772-4956-afd0-4d4a07bf8b80 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:50.770523 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe99418-4c69-46a8-9c51-991498367858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:50.771105 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe99418-4c69-46a8-9c51-991498367858 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:50.771379 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe99418-4c69-46a8-9c51-991498367858 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:50.771645 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe99418-4c69-46a8-9c51-991498367858 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:50.835211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe99418-4c69-46a8-9c51-991498367858 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:50.836627 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe99418-4c69-46a8-9c51-991498367858 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:50.922714 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:50.932451 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 572/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:55:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:55:50.939021 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5693d-b132-4e22-95f5-4853708be506 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:50.967524 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:51.033362 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5693d-b132-4e22-95f5-4853708be506 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:51.065826 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe155d-4b48-48ea-80e8-0b7ad1c42030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:51.066466 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe155d-4b48-48ea-80e8-0b7ad1c42030 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:51.066744 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe155d-4b48-48ea-80e8-0b7ad1c42030 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:51.067013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe155d-4b48-48ea-80e8-0b7ad1c42030 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:51.073563 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe155d-4b48-48ea-80e8-0b7ad1c42030 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:51.075054 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe155d-4b48-48ea-80e8-0b7ad1c42030 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:51.086916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-abbe155d-4b48-48ea-80e8-0b7ad1c42030 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 14 01:55:51.086916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:51.086916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 14 01:55:51.086916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 14 01:55:51.086916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:55:51.086916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:55:51.086916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:55:51.086916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:55:51.086916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 14 01:55:51.086916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:55:51.086916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:55:51.086916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:55:51.086916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:55:51.086916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:55:51.086916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:55:51.086916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:55:51.086916 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:55:51.088157 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:55:51.088157 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:55:51.088157 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:55:51.088157 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:55:51.088157 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:55:51.088157 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:55:51.088157 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:55:51.088157 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:55:51.088157 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:55:51.088157 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:55:51.088157 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:55:51.088157 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:55:51.088157 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:55:51.088157 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:55:51.088157 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:55:51.088157 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:55:51.089250 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:55:51.089250 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:55:51.089250 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:55:51.089250 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:55:51.089250 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:55:51.089250 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:55:51.089250 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:55:51.089250 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:55:51.089250 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:55:51.089250 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:55:51.089250 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 14 01:55:51.089250 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:55:51.089250 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 14 01:55:51.089250 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:55:51.089250 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 593/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Sep 14 01:55:50 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:55:51.094051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe99418-4c69-46a8-9c51-991498367858 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:51.192314 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe99418-4c69-46a8-9c51-991498367858 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:51.224897 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31263570-8212-49d7-abda-1e266a61aa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:51.225488 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31263570-8212-49d7-abda-1e266a61aa56 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:51.225764 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31263570-8212-49d7-abda-1e266a61aa56 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:51.226059 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31263570-8212-49d7-abda-1e266a61aa56 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:51.226687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31263570-8212-49d7-abda-1e266a61aa56 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:55:51.226984 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31263570-8212-49d7-abda-1e266a61aa56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:51.228333 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31263570-8212-49d7-abda-1e266a61aa56 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:51.235279 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 573/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Sep 14 01:55:51 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:53.331084 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81753769-5124-4ae9-b26e-cc49f0911c27 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:53.331637 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81753769-5124-4ae9-b26e-cc49f0911c27 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:53.331637 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81753769-5124-4ae9-b26e-cc49f0911c27 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:53.332922 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 594/1167] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:55:53.338064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4a416-b054-41df-8e57-f318f18fa5c7 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:53.338525 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4a416-b054-41df-8e57-f318f18fa5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:53.338623 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4a416-b054-41df-8e57-f318f18fa5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:53.356843 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-b8b4a416-b054-41df-8e57-f318f18fa5c7 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:53.358022 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-b8b4a416-b054-41df-8e57-f318f18fa5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:53.407106 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:53.423908 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 574/1168] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:55:53.690997 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81753769-5124-4ae9-b26e-cc49f0911c27 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:53.777853 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:53.788468 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81753769-5124-4ae9-b26e-cc49f0911c27 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:53.820526 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e662dde-a794-4780-b24d-8a6940f11317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:53.821113 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e662dde-a794-4780-b24d-8a6940f11317 None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:53.821388 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e662dde-a794-4780-b24d-8a6940f11317 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:53.821656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e662dde-a794-4780-b24d-8a6940f11317 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:53.822338 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e662dde-a794-4780-b24d-8a6940f11317 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:53.823683 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e662dde-a794-4780-b24d-8a6940f11317 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:53.838825 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 595/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Wed Sep 14 01:55:53 2022] POST /identity/v3/domains => generated 267 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:55:55.991904 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701f7c6-ca3d-4acf-be4c-6c34a885f274 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:55.992457 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701f7c6-ca3d-4acf-be4c-6c34a885f274 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:55.992457 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701f7c6-ca3d-4acf-be4c-6c34a885f274 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:55.993788 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 575/1170] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:55:55.999482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc165d-42ab-41a7-94a7-33562cc0d0f4 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:55.999878 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc165d-42ab-41a7-94a7-33562cc0d0f4 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:56.000030 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc165d-42ab-41a7-94a7-33562cc0d0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:56.018539 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-d0dc165d-42ab-41a7-94a7-33562cc0d0f4 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:56.019715 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-d0dc165d-42ab-41a7-94a7-33562cc0d0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:56.083276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 596/1171] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:55:56.352002 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701f7c6-ca3d-4acf-be4c-6c34a885f274 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:56.451893 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701f7c6-ca3d-4acf-be4c-6c34a885f274 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:56.483804 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b0da6-e3a0-401e-bffb-b705be052041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:56.484420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b0da6-e3a0-401e-bffb-b705be052041 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:56.484702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b0da6-e3a0-401e-bffb-b705be052041 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:56.484976 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b0da6-e3a0-401e-bffb-b705be052041 None admin] PATH_INFO: `/v3/domains/2b87692216de4f528a727d41bcc2d4d9` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:56.495048 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b0da6-e3a0-401e-bffb-b705be052041 None admin] RBAC: Authorizing `identity:get_domain(domain_id=2b87692216de4f528a727d41bcc2d4d9)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:56.496475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b0da6-e3a0-401e-bffb-b705be052041 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:56.498616 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 576/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Wed Sep 14 01:55:56 2022] GET /identity/v3/domains/2b87692216de4f528a727d41bcc2d4d9 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:56.504692 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc165d-42ab-41a7-94a7-33562cc0d0f4 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:56.584771 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:56.604035 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc165d-42ab-41a7-94a7-33562cc0d0f4 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:56.636914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15b031-36bb-41a0-a633-661b72daa1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:56.637520 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15b031-36bb-41a0-a633-661b72daa1bb None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:56.637800 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15b031-36bb-41a0-a633-661b72daa1bb None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:56.638115 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15b031-36bb-41a0-a633-661b72daa1bb None admin] PATH_INFO: `/v3/projects` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:56.639480 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15b031-36bb-41a0-a633-661b72daa1bb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:56.641788 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15b031-36bb-41a0-a633-661b72daa1bb None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:56.668520 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 597/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 14 01:55:56 2022] POST /identity/v3/projects => generated 361 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:55:58.822137 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641c700-395b-4fdf-a775-28e6003d9528 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:58.822658 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641c700-395b-4fdf-a775-28e6003d9528 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:58.822658 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641c700-395b-4fdf-a775-28e6003d9528 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:58.823846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 577/1174] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:55:58.829693 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f780d7-7285-439c-8581-864075cecb71 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:58.830137 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f780d7-7285-439c-8581-864075cecb71 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:58.830260 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f780d7-7285-439c-8581-864075cecb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:58.847828 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-56f780d7-7285-439c-8581-864075cecb71 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:55:58.848995 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-56f780d7-7285-439c-8581-864075cecb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:55:58.918189 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 598/1175] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:55:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:55:59.189885 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641c700-395b-4fdf-a775-28e6003d9528 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:59.235724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:59.288516 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641c700-395b-4fdf-a775-28e6003d9528 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:59.320598 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f10dc-5a76-4ebb-ab6e-74c35bfbc8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:59.321205 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f10dc-5a76-4ebb-ab6e-74c35bfbc8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:59.321478 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f10dc-5a76-4ebb-ab6e-74c35bfbc8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:59.321741 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f10dc-5a76-4ebb-ab6e-74c35bfbc8b2 None admin] PATH_INFO: `/v3/domains/2b87692216de4f528a727d41bcc2d4d9` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:59.322851 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f10dc-5a76-4ebb-ab6e-74c35bfbc8b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=2b87692216de4f528a727d41bcc2d4d9)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:59.324259 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f10dc-5a76-4ebb-ab6e-74c35bfbc8b2 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:59.325840 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 578/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Wed Sep 14 01:55:59 2022] GET /identity/v3/domains/2b87692216de4f528a727d41bcc2d4d9 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:55:59.331502 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f780d7-7285-439c-8581-864075cecb71 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:55:59.390009 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:55:59.428049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f780d7-7285-439c-8581-864075cecb71 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:55:59.460344 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da0ae6-16e1-4d08-878a-3b5f3c60b557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:55:59.460884 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da0ae6-16e1-4d08-878a-3b5f3c60b557 None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:55:59.461162 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da0ae6-16e1-4d08-878a-3b5f3c60b557 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:55:59.461432 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da0ae6-16e1-4d08-878a-3b5f3c60b557 None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:55:59.462107 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da0ae6-16e1-4d08-878a-3b5f3c60b557 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:55:59.463528 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da0ae6-16e1-4d08-878a-3b5f3c60b557 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:55:59.484125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 599/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 14 01:55:59 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:56:01.626256 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e69f8-5dc8-4334-82ab-c4a5e4cddd77 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:01.626797 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e69f8-5dc8-4334-82ab-c4a5e4cddd77 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:01.626797 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e69f8-5dc8-4334-82ab-c4a5e4cddd77 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:01.628106 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 579/1178] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:56:01.633610 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e314f-9720-4e92-81c1-63cd8e90dcc6 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:01.634109 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e314f-9720-4e92-81c1-63cd8e90dcc6 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:01.634230 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e314f-9720-4e92-81c1-63cd8e90dcc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:01.651895 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-a53e314f-9720-4e92-81c1-63cd8e90dcc6 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:56:01.653069 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-a53e314f-9720-4e92-81c1-63cd8e90dcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:56:01.717626 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 600/1179] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:56:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:56:01.986392 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e69f8-5dc8-4334-82ab-c4a5e4cddd77 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:56:02.086121 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e69f8-5dc8-4334-82ab-c4a5e4cddd77 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:56:02.098285 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:56:02.118997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3b825-29b0-45fe-b2f6-7594b084f89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:56:02.119622 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3b825-29b0-45fe-b2f6-7594b084f89e None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:02.119900 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3b825-29b0-45fe-b2f6-7594b084f89e None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:02.120169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3b825-29b0-45fe-b2f6-7594b084f89e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:02.140579 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a3b825-29b0-45fe-b2f6-7594b084f89e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:56:02.145362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-77a3b825-29b0-45fe-b2f6-7594b084f89e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:56:02.145362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:56:02.145362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:56:02.145362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:56:02.145362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:56:02.145362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:56:02.145362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:56:02.145362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:56:02.145362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:56:02.145362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:56:02.145362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:56:02.145362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:56:02.145362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:56:02.145362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:56:02.145362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:56:02.145362 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:56:02.146513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:56:02.146513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:56:02.146513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:56:02.146513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:56:02.146513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:56:02.146513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:56:02.146513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:56:02.146513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:56:02.146513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:56:02.146513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:56:02.146513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:56:02.146513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:56:02.146513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:56:02.146513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:56:02.146513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:56:02.146513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:56:02.147724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:56:02.147724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:56:02.147724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:56:02.147724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:56:02.147724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:56:02.147724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:56:02.147724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:56:02.147724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:56:02.147724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:56:02.147724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:56:02.147724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:56:02.147724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:56:02.147724 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 580/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:56:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:56:02.152572 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e314f-9720-4e92-81c1-63cd8e90dcc6 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:56:02.198991 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:56:02.248144 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e314f-9720-4e92-81c1-63cd8e90dcc6 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:56:02.280046 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aac28b-0eef-4bb6-8325-a511f0508f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:56:02.280645 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aac28b-0eef-4bb6-8325-a511f0508f3c None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:02.280924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aac28b-0eef-4bb6-8325-a511f0508f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:02.281197 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aac28b-0eef-4bb6-8325-a511f0508f3c None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:02.281849 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aac28b-0eef-4bb6-8325-a511f0508f3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:56:02.282176 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aac28b-0eef-4bb6-8325-a511f0508f3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:56:02.283541 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aac28b-0eef-4bb6-8325-a511f0508f3c None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:56:02.291193 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 601/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:56:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:56:02.297591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3b825-29b0-45fe-b2f6-7594b084f89e None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:56:02.393382 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3b825-29b0-45fe-b2f6-7594b084f89e None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:56:02.425751 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dc421-7d1f-4dde-a8d3-6dc6b5ed592d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:56:02.426400 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dc421-7d1f-4dde-a8d3-6dc6b5ed592d None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:02.426676 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dc421-7d1f-4dde-a8d3-6dc6b5ed592d None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:02.426942 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dc421-7d1f-4dde-a8d3-6dc6b5ed592d None admin] PATH_INFO: `/v3/users/ecb88292dd77424e9b8ba24e7c60af02` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:02.439982 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dc421-7d1f-4dde-a8d3-6dc6b5ed592d None admin] RBAC: Authorizing `identity:get_user(user_id=ecb88292dd77424e9b8ba24e7c60af02)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:56:02.441410 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dc421-7d1f-4dde-a8d3-6dc6b5ed592d None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:56:02.443022 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 581/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:56:02 2022] GET /identity/v3/users/ecb88292dd77424e9b8ba24e7c60af02 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:56:02.448899 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aac28b-0eef-4bb6-8325-a511f0508f3c None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:56:02.544124 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aac28b-0eef-4bb6-8325-a511f0508f3c None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:56:02.554330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:56:02.576475 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cabaa63-9993-4d9f-8964-0f72f88395ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:56:02.577062 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabaa63-9993-4d9f-8964-0f72f88395ef None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:02.577338 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabaa63-9993-4d9f-8964-0f72f88395ef None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:02.577608 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabaa63-9993-4d9f-8964-0f72f88395ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:02.640990 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cabaa63-9993-4d9f-8964-0f72f88395ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:56:02.642447 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cabaa63-9993-4d9f-8964-0f72f88395ef None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:56:02.734752 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 602/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:56:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:56:02.740535 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dc421-7d1f-4dde-a8d3-6dc6b5ed592d None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:56:02.770939 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:56:02.837100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dc421-7d1f-4dde-a8d3-6dc6b5ed592d None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:56:02.871364 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccbcda-4468-4bbf-9a8f-1aa2a75d807b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:56:02.871987 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccbcda-4468-4bbf-9a8f-1aa2a75d807b None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:02.872277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccbcda-4468-4bbf-9a8f-1aa2a75d807b None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:02.873103 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccbcda-4468-4bbf-9a8f-1aa2a75d807b None admin] PATH_INFO: `/v3/projects/569a63547a0b42c2bfb04e2912c9a8a8` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:02.874243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccbcda-4468-4bbf-9a8f-1aa2a75d807b None admin] RBAC: Authorizing `identity:get_project(project_id=569a63547a0b42c2bfb04e2912c9a8a8)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:56:02.875666 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccbcda-4468-4bbf-9a8f-1aa2a75d807b None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:56:02.877288 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 582/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:56:02 2022] GET /identity/v3/projects/569a63547a0b42c2bfb04e2912c9a8a8 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:56:02.883943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cabaa63-9993-4d9f-8964-0f72f88395ef None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:56:02.895246 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:56:02.978552 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cabaa63-9993-4d9f-8964-0f72f88395ef None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:56:03.011402 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d56d7c-3bc8-46e5-9604-97dd0cdac0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:56:03.011983 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d56d7c-3bc8-46e5-9604-97dd0cdac0d2 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:03.012257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d56d7c-3bc8-46e5-9604-97dd0cdac0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:03.012522 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d56d7c-3bc8-46e5-9604-97dd0cdac0d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:03.013245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d56d7c-3bc8-46e5-9604-97dd0cdac0d2 None admin] RBAC: Adding query filter params (role.id=aa420c2fb1ab4bb3b05b004a4d89779a, scope.project.id=569a63547a0b42c2bfb04e2912c9a8a8, user.id=ecb88292dd77424e9b8ba24e7c60af02) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:56:03.013544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d56d7c-3bc8-46e5-9604-97dd0cdac0d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:56:03.014975 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d56d7c-3bc8-46e5-9604-97dd0cdac0d2 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:56:03.020125 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 603/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:56:02 2022] GET /identity/v3/role_assignments?user.id=ecb88292dd77424e9b8ba24e7c60af02&scope.project.id=569a63547a0b42c2bfb04e2912c9a8a8&role.id=aa420c2fb1ab4bb3b05b004a4d89779a => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:56:05.100123 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820362b-3231-4781-ba40-e52169165e90 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:05.100653 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820362b-3231-4781-ba40-e52169165e90 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:05.100653 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820362b-3231-4781-ba40-e52169165e90 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:05.102025 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 583/1186] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:56:05.107291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf932ca2-7040-4b2f-89db-bde3aa8d42a4 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:05.107728 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf932ca2-7040-4b2f-89db-bde3aa8d42a4 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:05.107824 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf932ca2-7040-4b2f-89db-bde3aa8d42a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:05.125414 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-cf932ca2-7040-4b2f-89db-bde3aa8d42a4 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:56:05.126601 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-cf932ca2-7040-4b2f-89db-bde3aa8d42a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:56:05.191968 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 604/1187] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:56:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:56:05.457206 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820362b-3231-4781-ba40-e52169165e90 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:56:05.552350 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:56:05.557182 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820362b-3231-4781-ba40-e52169165e90 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:56:05.590255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8cd828-d9c7-43f1-88b7-7f74ca863cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:56:05.590867 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8cd828-d9c7-43f1-88b7-7f74ca863cce None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:05.591169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8cd828-d9c7-43f1-88b7-7f74ca863cce None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:05.591440 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8cd828-d9c7-43f1-88b7-7f74ca863cce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:05.601232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8cd828-d9c7-43f1-88b7-7f74ca863cce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:56:05.606074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-2a8cd828-d9c7-43f1-88b7-7f74ca863cce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:56:05.606074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:56:05.606074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:56:05.606074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:56:05.606074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:56:05.606074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:56:05.606074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:56:05.606074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:56:05.606074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:56:05.606074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:56:05.606074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:56:05.606074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:56:05.606074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:56:05.606074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:56:05.606074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:56:05.606074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:56:05.607194 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:56:05.607194 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:56:05.607194 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:56:05.607194 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:56:05.607194 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:56:05.607194 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:56:05.607194 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:56:05.607194 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:56:05.607194 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:56:05.607194 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:56:05.607194 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:56:05.607194 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:56:05.607194 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:56:05.607194 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:56:05.607194 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:56:05.607194 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:56:05.608240 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:56:05.608240 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:56:05.608240 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:56:05.608240 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:56:05.608240 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:56:05.608240 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:56:05.608240 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:56:05.608240 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:56:05.608240 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:56:05.608240 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:56:05.608240 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:56:05.608240 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:56:05.608240 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 584/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:56:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:56:05.613040 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf932ca2-7040-4b2f-89db-bde3aa8d42a4 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:56:05.616086 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:56:05.709779 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf932ca2-7040-4b2f-89db-bde3aa8d42a4 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:56:05.741556 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46db46-5d1d-498a-a3a0-8806ded35da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:56:05.742179 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46db46-5d1d-498a-a3a0-8806ded35da2 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:05.742459 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46db46-5d1d-498a-a3a0-8806ded35da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:05.742730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46db46-5d1d-498a-a3a0-8806ded35da2 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:05.743380 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46db46-5d1d-498a-a3a0-8806ded35da2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:56:05.743681 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46db46-5d1d-498a-a3a0-8806ded35da2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:56:05.745033 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46db46-5d1d-498a-a3a0-8806ded35da2 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:56:05.753259 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 605/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:56:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:56:05.759876 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8cd828-d9c7-43f1-88b7-7f74ca863cce None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:56:05.855188 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8cd828-d9c7-43f1-88b7-7f74ca863cce None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:56:05.887731 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e57db-4da7-4135-a1c2-7169f30494d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:56:05.888882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e57db-4da7-4135-a1c2-7169f30494d4 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:05.889162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e57db-4da7-4135-a1c2-7169f30494d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:05.889433 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e57db-4da7-4135-a1c2-7169f30494d4 None admin] PATH_INFO: `/v3/users/ecb88292dd77424e9b8ba24e7c60af02` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:05.890563 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e57db-4da7-4135-a1c2-7169f30494d4 None admin] RBAC: Authorizing `identity:get_user(user_id=ecb88292dd77424e9b8ba24e7c60af02)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:56:05.891980 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e57db-4da7-4135-a1c2-7169f30494d4 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:56:05.893513 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 585/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:56:05 2022] GET /identity/v3/users/ecb88292dd77424e9b8ba24e7c60af02 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:56:05.898996 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46db46-5d1d-498a-a3a0-8806ded35da2 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:56:05.961584 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:56:05.995041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46db46-5d1d-498a-a3a0-8806ded35da2 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:56:06.027261 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c4551-e879-4224-92ac-1a66cc3358d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:56:06.027869 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c4551-e879-4224-92ac-1a66cc3358d4 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:06.028147 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c4551-e879-4224-92ac-1a66cc3358d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:06.028425 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c4551-e879-4224-92ac-1a66cc3358d4 None admin] PATH_INFO: `/v3/projects/569a63547a0b42c2bfb04e2912c9a8a8` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:06.029518 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c4551-e879-4224-92ac-1a66cc3358d4 None admin] RBAC: Authorizing `identity:get_project(project_id=569a63547a0b42c2bfb04e2912c9a8a8)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:56:06.030977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c4551-e879-4224-92ac-1a66cc3358d4 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:56:06.032551 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 606/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:56:05 2022] GET /identity/v3/projects/569a63547a0b42c2bfb04e2912c9a8a8 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:56:06.038247 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e57db-4da7-4135-a1c2-7169f30494d4 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:56:06.065499 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:56:06.134805 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e57db-4da7-4135-a1c2-7169f30494d4 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:56:06.167284 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d0678-8bd5-4586-9570-5d6163eb5821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:56:06.167923 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d0678-8bd5-4586-9570-5d6163eb5821 None admin] REQUEST_METHOD: `PUT` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:06.168196 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d0678-8bd5-4586-9570-5d6163eb5821 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:06.168462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d0678-8bd5-4586-9570-5d6163eb5821 None admin] PATH_INFO: `/v3/projects/569a63547a0b42c2bfb04e2912c9a8a8/users/ecb88292dd77424e9b8ba24e7c60af02/roles/aa420c2fb1ab4bb3b05b004a4d89779a` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:06.170370 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d0678-8bd5-4586-9570-5d6163eb5821 None admin] RBAC: Authorizing `identity:create_grant(project_id=569a63547a0b42c2bfb04e2912c9a8a8, user_id=ecb88292dd77424e9b8ba24e7c60af02, role_id=aa420c2fb1ab4bb3b05b004a4d89779a)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:56:06.171775 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d0678-8bd5-4586-9570-5d6163eb5821 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:56:06.183539 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 586/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:56:06 2022] PUT /identity/v3/projects/569a63547a0b42c2bfb04e2912c9a8a8/users/ecb88292dd77424e9b8ba24e7c60af02/roles/aa420c2fb1ab4bb3b05b004a4d89779a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:56:08.266478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08511b27-36f4-4c85-a50c-7cccdde23754 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:08.267100 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08511b27-36f4-4c85-a50c-7cccdde23754 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:08.267100 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08511b27-36f4-4c85-a50c-7cccdde23754 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:08.268329 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 607/1193] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:56:08.273455 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00366f60-1c4c-497b-8d63-62037c81427a None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:08.273872 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00366f60-1c4c-497b-8d63-62037c81427a None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:08.274010 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00366f60-1c4c-497b-8d63-62037c81427a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:08.297007 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-00366f60-1c4c-497b-8d63-62037c81427a None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:56:08.298225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-00366f60-1c4c-497b-8d63-62037c81427a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:56:08.364729 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 587/1194] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:56:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:56:08.631716 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08511b27-36f4-4c85-a50c-7cccdde23754 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:56:08.731268 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08511b27-36f4-4c85-a50c-7cccdde23754 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:56:08.763875 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9fabf-a0ca-4885-834e-fe9c57661d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:56:08.764494 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9fabf-a0ca-4885-834e-fe9c57661d93 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:08.764774 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9fabf-a0ca-4885-834e-fe9c57661d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:08.765049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9fabf-a0ca-4885-834e-fe9c57661d93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:08.766858 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:56:08.775571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9fabf-a0ca-4885-834e-fe9c57661d93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:56:08.780874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-60d9fabf-a0ca-4885-834e-fe9c57661d93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:56:08.780874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:56:08.780874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:56:08.780874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:56:08.780874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:56:08.780874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:56:08.780874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:56:08.780874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:56:08.780874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:56:08.780874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:56:08.780874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:56:08.780874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:56:08.780874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:56:08.780874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:56:08.780874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:56:08.780874 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:56:08.782029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:56:08.782029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:56:08.782029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:56:08.782029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:56:08.782029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:56:08.782029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:56:08.782029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:56:08.782029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:56:08.782029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:56:08.782029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:56:08.782029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:56:08.782029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:56:08.782029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:56:08.782029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:56:08.782029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:56:08.782029 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:56:08.782970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:56:08.782970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:56:08.782970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:56:08.782970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:56:08.782970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:56:08.782970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:56:08.782970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:56:08.782970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:56:08.782970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:56:08.782970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:56:08.782970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:56:08.782970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:56:08.782970 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 608/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:56:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:56:08.787998 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00366f60-1c4c-497b-8d63-62037c81427a None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:56:08.805441 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:56:08.884888 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00366f60-1c4c-497b-8d63-62037c81427a None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:56:08.917789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215de486-64d0-4af8-8579-cfdd78cf58af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:56:08.918411 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215de486-64d0-4af8-8579-cfdd78cf58af None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:08.918691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215de486-64d0-4af8-8579-cfdd78cf58af None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:08.919474 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215de486-64d0-4af8-8579-cfdd78cf58af None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:08.920134 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215de486-64d0-4af8-8579-cfdd78cf58af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:56:08.920435 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215de486-64d0-4af8-8579-cfdd78cf58af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:56:08.921792 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215de486-64d0-4af8-8579-cfdd78cf58af None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:56:08.929575 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 588/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:56:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:56:08.938431 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9fabf-a0ca-4885-834e-fe9c57661d93 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:56:09.034656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9fabf-a0ca-4885-834e-fe9c57661d93 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:56:09.068093 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d2bbc-c598-4d86-bf3a-ca0c3744ef6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:56:09.068704 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d2bbc-c598-4d86-bf3a-ca0c3744ef6e None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:09.068986 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d2bbc-c598-4d86-bf3a-ca0c3744ef6e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:09.069263 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d2bbc-c598-4d86-bf3a-ca0c3744ef6e None admin] PATH_INFO: `/v3/users/ecb88292dd77424e9b8ba24e7c60af02` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:09.070400 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d2bbc-c598-4d86-bf3a-ca0c3744ef6e None admin] RBAC: Authorizing `identity:get_user(user_id=ecb88292dd77424e9b8ba24e7c60af02)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:56:09.071830 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d2bbc-c598-4d86-bf3a-ca0c3744ef6e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:56:09.073362 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 609/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 14 01:56:08 2022] GET /identity/v3/users/ecb88292dd77424e9b8ba24e7c60af02 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:56:09.079053 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215de486-64d0-4af8-8579-cfdd78cf58af None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:56:09.154531 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:56:09.175839 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215de486-64d0-4af8-8579-cfdd78cf58af None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:56:09.207694 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8ebca-0ba1-469a-b665-11e5aa508d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:56:09.208277 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8ebca-0ba1-469a-b665-11e5aa508d69 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:09.208556 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8ebca-0ba1-469a-b665-11e5aa508d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:09.208825 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8ebca-0ba1-469a-b665-11e5aa508d69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:09.274655 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8ebca-0ba1-469a-b665-11e5aa508d69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:56:09.276100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8ebca-0ba1-469a-b665-11e5aa508d69 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:56:09.348642 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:56:09.369691 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 589/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:56:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:56:09.376162 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d2bbc-c598-4d86-bf3a-ca0c3744ef6e None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:56:09.436330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:56:09.473099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d2bbc-c598-4d86-bf3a-ca0c3744ef6e None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:56:09.505544 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c0938-44ac-4b73-92cf-e9369c923edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:56:09.506182 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c0938-44ac-4b73-92cf-e9369c923edb None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:09.506457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c0938-44ac-4b73-92cf-e9369c923edb None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:09.506725 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c0938-44ac-4b73-92cf-e9369c923edb None admin] PATH_INFO: `/v3/projects/569a63547a0b42c2bfb04e2912c9a8a8` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:09.507804 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c0938-44ac-4b73-92cf-e9369c923edb None admin] RBAC: Authorizing `identity:get_project(project_id=569a63547a0b42c2bfb04e2912c9a8a8)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:56:09.509228 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c0938-44ac-4b73-92cf-e9369c923edb None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:56:09.510844 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 610/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 14 01:56:09 2022] GET /identity/v3/projects/569a63547a0b42c2bfb04e2912c9a8a8 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:56:09.517525 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8ebca-0ba1-469a-b665-11e5aa508d69 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:56:09.613021 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8ebca-0ba1-469a-b665-11e5aa508d69 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:56:09.645605 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a87a59-d67c-4f28-ac70-51a7b8ea601a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:56:09.646221 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a87a59-d67c-4f28-ac70-51a7b8ea601a None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:56:09.646496 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a87a59-d67c-4f28-ac70-51a7b8ea601a None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:56:09.646761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a87a59-d67c-4f28-ac70-51a7b8ea601a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:56:09.647467 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a87a59-d67c-4f28-ac70-51a7b8ea601a None admin] RBAC: Adding query filter params (role.id=aa420c2fb1ab4bb3b05b004a4d89779a, scope.project.id=569a63547a0b42c2bfb04e2912c9a8a8, user.id=ecb88292dd77424e9b8ba24e7c60af02) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:56:09.647765 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a87a59-d67c-4f28-ac70-51a7b8ea601a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:56:09.649163 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a87a59-d67c-4f28-ac70-51a7b8ea601a None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:56:09.654715 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 590/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:56:09 2022] GET /identity/v3/role_assignments?user.id=ecb88292dd77424e9b8ba24e7c60af02&scope.project.id=569a63547a0b42c2bfb04e2912c9a8a8&role.id=aa420c2fb1ab4bb3b05b004a4d89779a => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:56:19.501312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=35 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:56:19.652275 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=79 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:05.397733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215708ca-6a58-43fd-abee-bff6153fedc6 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:05.398330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215708ca-6a58-43fd-abee-bff6153fedc6 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:05.398330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215708ca-6a58-43fd-abee-bff6153fedc6 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:05.399846 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 611/1201] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:57:05.405326 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab18e3-b1ef-4bd2-bc23-39266f47d4ce None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:05.405799 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab18e3-b1ef-4bd2-bc23-39266f47d4ce None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:05.405886 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab18e3-b1ef-4bd2-bc23-39266f47d4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:05.428306 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-aaab18e3-b1ef-4bd2-bc23-39266f47d4ce None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:57:05.429532 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-aaab18e3-b1ef-4bd2-bc23-39266f47d4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:57:05.521252 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 591/1202] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:57:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:57:05.828826 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215708ca-6a58-43fd-abee-bff6153fedc6 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:05.969936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215708ca-6a58-43fd-abee-bff6153fedc6 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:06.017023 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83e48c-d952-4b00-b69b-2789c43a6e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:06.017734 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83e48c-d952-4b00-b69b-2789c43a6e6d None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:06.018079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83e48c-d952-4b00-b69b-2789c43a6e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:06.018379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83e48c-d952-4b00-b69b-2789c43a6e6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:06.036503 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83e48c-d952-4b00-b69b-2789c43a6e6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:06.038567 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83e48c-d952-4b00-b69b-2789c43a6e6d None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:06.060775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-4b83e48c-d952-4b00-b69b-2789c43a6e6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:06.060775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:06.060775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:06.060775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:06.060775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:06.060775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:06.060775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:06.060775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:06.060775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:06.060775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:06.060775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:06.060775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:06.060775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:57:06.060775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:57:06.060775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:57:06.060775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:57:06.061906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:06.061906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:06.061906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:06.061906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:06.061906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:06.061906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:06.061906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:06.061906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:06.061906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:06.061906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:06.061906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:06.061906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:06.061906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:06.061906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:06.061906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:06.061906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:06.061906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:57:06.062945 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:57:06.062945 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:06.062945 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:57:06.062945 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 612/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:57:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:06.068178 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab18e3-b1ef-4bd2-bc23-39266f47d4ce None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:06.205275 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab18e3-b1ef-4bd2-bc23-39266f47d4ce None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:06.253117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:06.261920 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede89e0-82c5-4b92-b17e-170b2e57d342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:06.262657 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede89e0-82c5-4b92-b17e-170b2e57d342 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:06.262931 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede89e0-82c5-4b92-b17e-170b2e57d342 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:06.263322 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede89e0-82c5-4b92-b17e-170b2e57d342 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:06.264247 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede89e0-82c5-4b92-b17e-170b2e57d342 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:06.264495 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede89e0-82c5-4b92-b17e-170b2e57d342 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:06.266350 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede89e0-82c5-4b92-b17e-170b2e57d342 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:06.290043 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 592/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:57:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:06.295825 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83e48c-d952-4b00-b69b-2789c43a6e6d None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:06.357379 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:06.423662 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83e48c-d952-4b00-b69b-2789c43a6e6d None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:06.465454 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ba070-3868-4a35-8cdc-57297452275e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:06.466013 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ba070-3868-4a35-8cdc-57297452275e None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:06.466401 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ba070-3868-4a35-8cdc-57297452275e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:06.466673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ba070-3868-4a35-8cdc-57297452275e None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:06.467337 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ba070-3868-4a35-8cdc-57297452275e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:06.469178 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ba070-3868-4a35-8cdc-57297452275e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:06.494973 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 613/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 14 01:57:06 2022] POST /identity/v3/users => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:57:08.914027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f5e34-04bb-49c5-bda3-828c597cc394 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:08.916550 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f5e34-04bb-49c5-bda3-828c597cc394 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:08.916550 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f5e34-04bb-49c5-bda3-828c597cc394 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:08.918619 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 593/1206] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:57:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:57:08.924345 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c40b2-473c-4ba4-81d2-dc5c28159d76 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:08.924841 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c40b2-473c-4ba4-81d2-dc5c28159d76 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:08.924955 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c40b2-473c-4ba4-81d2-dc5c28159d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:08.946922 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-5a6c40b2-473c-4ba4-81d2-dc5c28159d76 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:57:08.948949 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-5a6c40b2-473c-4ba4-81d2-dc5c28159d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:57:09.028063 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 614/1207] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:57:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:57:09.317301 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f5e34-04bb-49c5-bda3-828c597cc394 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:09.436581 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f5e34-04bb-49c5-bda3-828c597cc394 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:09.481712 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71b1e2-46bd-4d1b-95a7-214df6c48e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:09.482478 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71b1e2-46bd-4d1b-95a7-214df6c48e0f None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:09.482789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71b1e2-46bd-4d1b-95a7-214df6c48e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:09.483136 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71b1e2-46bd-4d1b-95a7-214df6c48e0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:09.495899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71b1e2-46bd-4d1b-95a7-214df6c48e0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:57:09.501091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-6c71b1e2-46bd-4d1b-95a7-214df6c48e0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:57:09.501091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:09.501091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:09.501091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:09.501091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:09.501091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:09.501091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:09.501091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:09.501091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:09.501091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:09.501091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:09.501091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:09.501091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:57:09.501091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:57:09.501091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:57:09.501091 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:57:09.503418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:57:09.503418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:57:09.503418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:57:09.503418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:57:09.503418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:09.503418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:09.503418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:09.503418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:09.503418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:09.503418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:09.503418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:09.503418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:09.503418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:09.503418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:09.503418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:09.503418 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:09.504630 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:09.504630 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:09.504630 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:09.504630 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:09.504630 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:57:09.504630 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:57:09.504630 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:57:09.504630 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:57:09.504630 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:57:09.504630 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:57:09.504630 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:57:09.504630 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:09.504630 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 594/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:57:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:09.508703 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c40b2-473c-4ba4-81d2-dc5c28159d76 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:09.554960 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:09.623740 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c40b2-473c-4ba4-81d2-dc5c28159d76 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:09.663812 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9013a03-6234-4171-9153-a493357a5181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:09.665219 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9013a03-6234-4171-9153-a493357a5181 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:09.665219 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9013a03-6234-4171-9153-a493357a5181 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:09.665419 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9013a03-6234-4171-9153-a493357a5181 None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:09.665913 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9013a03-6234-4171-9153-a493357a5181 None admin] RBAC: Adding query filter params (name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:09.666269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9013a03-6234-4171-9153-a493357a5181 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:09.668202 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9013a03-6234-4171-9153-a493357a5181 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:09.681714 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 615/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 14 01:57:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:09.682953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71b1e2-46bd-4d1b-95a7-214df6c48e0f None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:09.741698 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:09.811988 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71b1e2-46bd-4d1b-95a7-214df6c48e0f None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:09.849710 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c50eb-080b-4281-a8ae-12d14a23dbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:09.850364 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c50eb-080b-4281-a8ae-12d14a23dbf8 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:09.850687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c50eb-080b-4281-a8ae-12d14a23dbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:09.851008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c50eb-080b-4281-a8ae-12d14a23dbf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:09.863382 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c50eb-080b-4281-a8ae-12d14a23dbf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:09.864881 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c50eb-080b-4281-a8ae-12d14a23dbf8 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:09.882784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-a89c50eb-080b-4281-a8ae-12d14a23dbf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:09.882784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:09.882784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:09.882784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:09.882784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:09.882784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:09.882784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:09.882784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:09.882784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:09.882784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:09.882784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:09.882784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:09.882784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:57:09.882784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:57:09.882784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:57:09.882784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:57:09.883814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:09.883814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:09.883814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:09.883814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:09.883814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:09.883814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:09.883814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:09.883814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:09.883814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:09.883814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:09.883814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:09.883814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:09.883814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:09.883814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:09.883814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:09.883814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:09.883814 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:57:09.884772 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:57:09.884772 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:09.884772 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:09.890584 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 595/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:57:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:09.890744 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9013a03-6234-4171-9153-a493357a5181 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:10.002080 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:10.006883 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9013a03-6234-4171-9153-a493357a5181 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:10.056551 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660eb9bf-3a5f-4567-8c57-6ba4e9376aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:10.058007 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660eb9bf-3a5f-4567-8c57-6ba4e9376aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:10.058488 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660eb9bf-3a5f-4567-8c57-6ba4e9376aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:10.058864 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660eb9bf-3a5f-4567-8c57-6ba4e9376aa7 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:10.059958 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660eb9bf-3a5f-4567-8c57-6ba4e9376aa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:10.060455 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660eb9bf-3a5f-4567-8c57-6ba4e9376aa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:10.062213 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660eb9bf-3a5f-4567-8c57-6ba4e9376aa7 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:10.074893 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 616/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:57:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:10.080647 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c50eb-080b-4281-a8ae-12d14a23dbf8 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:10.196453 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c50eb-080b-4281-a8ae-12d14a23dbf8 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:10.198884 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:10.234712 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56d3f4-918a-4e44-bfa6-03939936ca02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:10.235734 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56d3f4-918a-4e44-bfa6-03939936ca02 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:10.236008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56d3f4-918a-4e44-bfa6-03939936ca02 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:10.236274 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56d3f4-918a-4e44-bfa6-03939936ca02 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:10.245269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e56d3f4-918a-4e44-bfa6-03939936ca02 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:10.246744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e56d3f4-918a-4e44-bfa6-03939936ca02 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:10.258784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-2e56d3f4-918a-4e44-bfa6-03939936ca02 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:57:10.258784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:10.258784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:10.258784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:10.258784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:10.258784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:10.258784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:10.258784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:10.258784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:10.258784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:10.258784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:10.258784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:10.258784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:57:10.258784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:57:10.258784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:57:10.258784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:57:10.259828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:10.259828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:10.259828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:57:10.259828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:57:10.259828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:57:10.259828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:57:10.259828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:10.259828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:10.259828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:10.259828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:10.259828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:10.259828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:10.259828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:10.259828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:10.259828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:10.259828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:10.259828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:10.259828 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:10.265681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:10.265681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:10.265681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:57:10.265681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:57:10.265681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:57:10.265681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:57:10.265681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:57:10.265681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:57:10.265681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:57:10.265681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:10.265681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 596/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:57:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:10.266357 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660eb9bf-3a5f-4567-8c57-6ba4e9376aa7 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:10.395889 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660eb9bf-3a5f-4567-8c57-6ba4e9376aa7 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:10.431711 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ed12c-3cfc-4ade-b5cf-8a7fafe1b9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:10.432349 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ed12c-3cfc-4ade-b5cf-8a7fafe1b9ff None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:10.432657 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ed12c-3cfc-4ade-b5cf-8a7fafe1b9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:10.432931 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ed12c-3cfc-4ade-b5cf-8a7fafe1b9ff None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:10.433653 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ed12c-3cfc-4ade-b5cf-8a7fafe1b9ff None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:10.433959 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ed12c-3cfc-4ade-b5cf-8a7fafe1b9ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:10.435423 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ed12c-3cfc-4ade-b5cf-8a7fafe1b9ff None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:10.454703 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 617/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Sep 14 01:57:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:10.460232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56d3f4-918a-4e44-bfa6-03939936ca02 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:10.574624 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56d3f4-918a-4e44-bfa6-03939936ca02 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:10.613016 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5022f0-1c9b-49a3-af9b-a0083f809ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:10.613352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5022f0-1c9b-49a3-af9b-a0083f809ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:10.613674 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5022f0-1c9b-49a3-af9b-a0083f809ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:10.613948 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5022f0-1c9b-49a3-af9b-a0083f809ef4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:10.650286 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:10.708273 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5022f0-1c9b-49a3-af9b-a0083f809ef4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:10.709753 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5022f0-1c9b-49a3-af9b-a0083f809ef4 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:10.829787 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 597/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:57:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:57:10.836207 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ed12c-3cfc-4ade-b5cf-8a7fafe1b9ff None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:10.857322 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:10.940311 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ed12c-3cfc-4ade-b5cf-8a7fafe1b9ff None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:10.976701 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a434cfd-948c-4d9d-b23e-3d503f73e063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:10.977267 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a434cfd-948c-4d9d-b23e-3d503f73e063 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:10.977530 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a434cfd-948c-4d9d-b23e-3d503f73e063 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:10.977801 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a434cfd-948c-4d9d-b23e-3d503f73e063 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:10.988863 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a434cfd-948c-4d9d-b23e-3d503f73e063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:10.990459 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a434cfd-948c-4d9d-b23e-3d503f73e063 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:11.006656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-9a434cfd-948c-4d9d-b23e-3d503f73e063 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:11.006656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:11.006656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:11.006656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:11.006656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:11.006656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:11.006656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:11.006656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:11.006656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:11.006656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:11.006656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:11.006656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:11.006656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:57:11.006656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:57:11.006656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:57:11.006656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:57:11.008370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:11.008370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:11.008370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:11.008370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:11.008370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:11.008370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:11.008370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:11.008370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:11.008370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:11.008370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:11.008370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:11.008370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:11.008370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:11.008370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:11.008370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:11.008370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:11.008370 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:57:11.009614 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:57:11.009614 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:11.009614 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:57:11.009614 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 618/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:57:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:57:11.013928 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5022f0-1c9b-49a3-af9b-a0083f809ef4 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:11.080488 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:11.121593 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5022f0-1c9b-49a3-af9b-a0083f809ef4 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:11.157771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2691982-26da-4d92-bbda-f0befd863fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:11.158414 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2691982-26da-4d92-bbda-f0befd863fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:11.158698 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2691982-26da-4d92-bbda-f0befd863fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:11.158964 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2691982-26da-4d92-bbda-f0befd863fa5 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:11.159605 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2691982-26da-4d92-bbda-f0befd863fa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:11.159914 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2691982-26da-4d92-bbda-f0befd863fa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:11.161296 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2691982-26da-4d92-bbda-f0befd863fa5 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:11.174521 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 598/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:57:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:11.181186 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a434cfd-948c-4d9d-b23e-3d503f73e063 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:11.258711 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:11.293295 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a434cfd-948c-4d9d-b23e-3d503f73e063 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:11.326881 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2769984-d1f5-4203-ac3a-de194160de25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:11.327506 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2769984-d1f5-4203-ac3a-de194160de25 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:11.327786 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2769984-d1f5-4203-ac3a-de194160de25 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:11.328050 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2769984-d1f5-4203-ac3a-de194160de25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:11.334499 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2769984-d1f5-4203-ac3a-de194160de25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:11.335953 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2769984-d1f5-4203-ac3a-de194160de25 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:11.347702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-e2769984-d1f5-4203-ac3a-de194160de25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:57:11.347702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:11.347702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:11.347702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:11.347702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:11.347702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:11.347702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:11.347702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:11.347702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:11.347702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:11.347702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:11.347702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:11.347702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:57:11.347702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:57:11.347702 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:57:11.348793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:57:11.348793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:11.348793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:11.348793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:11.348793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:11.348793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:11.348793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:11.348793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:11.348793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:11.348793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:11.348793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:11.348793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:11.348793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:11.348793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:11.348793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:11.348793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:11.348793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:11.349961 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:57:11.349961 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:57:11.349961 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:57:11.349961 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:57:11.349961 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:57:11.349961 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:57:11.349961 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:57:11.349961 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:57:11.349961 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 619/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:57:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:11.355083 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2691982-26da-4d92-bbda-f0befd863fa5 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:11.459873 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2691982-26da-4d92-bbda-f0befd863fa5 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:11.484320 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:11.495971 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96b275-5d04-43eb-befe-7dcde496a347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:11.496577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96b275-5d04-43eb-befe-7dcde496a347 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:11.496856 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96b275-5d04-43eb-befe-7dcde496a347 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:11.497126 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96b275-5d04-43eb-befe-7dcde496a347 None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:11.497809 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d96b275-5d04-43eb-befe-7dcde496a347 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:11.498146 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d96b275-5d04-43eb-befe-7dcde496a347 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:11.499571 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d96b275-5d04-43eb-befe-7dcde496a347 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:11.516794 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 599/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:57:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:11.518030 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2769984-d1f5-4203-ac3a-de194160de25 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:11.619439 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2769984-d1f5-4203-ac3a-de194160de25 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:11.650998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:11.657524 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bb7b62-c57e-46b8-a311-6c34c778a53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:11.658191 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb7b62-c57e-46b8-a311-6c34c778a53e None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:11.658471 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb7b62-c57e-46b8-a311-6c34c778a53e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:11.658741 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb7b62-c57e-46b8-a311-6c34c778a53e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:11.659464 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bb7b62-c57e-46b8-a311-6c34c778a53e None admin] RBAC: Adding query filter params (role.id=e962a2a280224f9fbcd0fdc02262993d, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=4da8e712cd9b454fa035f93785775fd8) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:11.659772 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bb7b62-c57e-46b8-a311-6c34c778a53e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:11.661194 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bb7b62-c57e-46b8-a311-6c34c778a53e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:11.678023 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 620/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:57:11 2022] GET /identity/v3/role_assignments?user.id=4da8e712cd9b454fa035f93785775fd8&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=e962a2a280224f9fbcd0fdc02262993d => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:13.978469 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da1a9a-3a04-4dfd-88b9-743d2b1d3040 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:13.979044 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da1a9a-3a04-4dfd-88b9-743d2b1d3040 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:13.979044 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da1a9a-3a04-4dfd-88b9-743d2b1d3040 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:13.980608 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 600/1220] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:57:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:57:13.985861 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e897cc-d345-4cf9-b635-9b8ee4d4fcc5 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:13.986422 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e897cc-d345-4cf9-b635-9b8ee4d4fcc5 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:13.986526 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e897cc-d345-4cf9-b635-9b8ee4d4fcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:14.007337 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-57e897cc-d345-4cf9-b635-9b8ee4d4fcc5 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:57:14.008591 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-57e897cc-d345-4cf9-b635-9b8ee4d4fcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:57:14.088402 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 621/1221] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:57:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:57:14.366509 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da1a9a-3a04-4dfd-88b9-743d2b1d3040 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:14.474211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da1a9a-3a04-4dfd-88b9-743d2b1d3040 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:14.512712 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d909855-35b1-4ad1-907e-928d4cefbe6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:14.513431 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d909855-35b1-4ad1-907e-928d4cefbe6f None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:14.513736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d909855-35b1-4ad1-907e-928d4cefbe6f None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:14.514102 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d909855-35b1-4ad1-907e-928d4cefbe6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:14.525222 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d909855-35b1-4ad1-907e-928d4cefbe6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:57:14.530953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-9d909855-35b1-4ad1-907e-928d4cefbe6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:57:14.530953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:14.530953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:14.530953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:14.530953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:14.530953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:14.530953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:14.530953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:14.530953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:14.530953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:14.530953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:14.530953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:14.530953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:57:14.530953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:57:14.530953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:57:14.530953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:57:14.532246 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:57:14.532246 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:57:14.532246 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:57:14.532246 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:57:14.532246 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:14.532246 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:14.532246 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:14.532246 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:14.532246 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:14.532246 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:14.532246 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:14.532246 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:14.532246 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:14.532246 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:14.532246 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:14.532246 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:14.533547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:14.533547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:14.533547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:14.533547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:14.533547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:57:14.533547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:57:14.533547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:57:14.533547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:57:14.533547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:57:14.533547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:57:14.533547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:57:14.533547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:14.533547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 601/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:57:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:57:14.538725 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e897cc-d345-4cf9-b635-9b8ee4d4fcc5 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:14.652152 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e897cc-d345-4cf9-b635-9b8ee4d4fcc5 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:14.680573 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:14.692851 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba640a-7c70-4a8b-a1b2-982daf760f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:14.693539 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba640a-7c70-4a8b-a1b2-982daf760f9a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:14.693826 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba640a-7c70-4a8b-a1b2-982daf760f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:14.694174 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba640a-7c70-4a8b-a1b2-982daf760f9a None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:14.694873 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba640a-7c70-4a8b-a1b2-982daf760f9a None admin] RBAC: Adding query filter params (name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:14.695215 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba640a-7c70-4a8b-a1b2-982daf760f9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:14.696704 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba640a-7c70-4a8b-a1b2-982daf760f9a None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:14.704671 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 622/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 14 01:57:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:14.710563 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d909855-35b1-4ad1-907e-928d4cefbe6f None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:14.759452 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:14.822674 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d909855-35b1-4ad1-907e-928d4cefbe6f None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:14.865881 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff803084-82b0-4d87-acf7-f555e9a44868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:14.866534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff803084-82b0-4d87-acf7-f555e9a44868 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:14.866810 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff803084-82b0-4d87-acf7-f555e9a44868 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:14.867165 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff803084-82b0-4d87-acf7-f555e9a44868 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:14.879547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff803084-82b0-4d87-acf7-f555e9a44868 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:14.881004 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff803084-82b0-4d87-acf7-f555e9a44868 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:14.896356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-ff803084-82b0-4d87-acf7-f555e9a44868 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:14.896356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:14.896356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:14.896356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:14.896356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:14.896356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:14.896356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:14.896356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:14.896356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:14.896356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:14.896356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:14.896356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:14.896356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:57:14.896356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:57:14.896356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:57:14.896356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:57:14.897641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:14.897641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:14.897641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:14.897641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:14.897641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:14.897641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:14.897641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:14.897641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:14.897641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:14.897641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:14.897641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:14.897641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:14.897641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:14.897641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:14.897641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:14.897641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:14.897641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:57:14.898878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:57:14.898878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:14.898878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:14.898878 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 602/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:57:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:57:14.904036 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba640a-7c70-4a8b-a1b2-982daf760f9a None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:15.014039 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba640a-7c70-4a8b-a1b2-982daf760f9a None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:15.055753 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4dc6e9-fccb-438e-bf87-1a21c814ad72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:15.056466 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dc6e9-fccb-438e-bf87-1a21c814ad72 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:15.056751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dc6e9-fccb-438e-bf87-1a21c814ad72 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:15.057024 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dc6e9-fccb-438e-bf87-1a21c814ad72 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:15.057713 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4dc6e9-fccb-438e-bf87-1a21c814ad72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:15.058041 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4dc6e9-fccb-438e-bf87-1a21c814ad72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:15.059576 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4dc6e9-fccb-438e-bf87-1a21c814ad72 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:15.072560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 623/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:57:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:15.079274 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff803084-82b0-4d87-acf7-f555e9a44868 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:15.186148 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:15.193185 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff803084-82b0-4d87-acf7-f555e9a44868 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:15.231428 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c00e86-8f00-434a-9752-b57582aeadbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:15.232115 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c00e86-8f00-434a-9752-b57582aeadbc None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:15.232396 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c00e86-8f00-434a-9752-b57582aeadbc None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:15.232704 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c00e86-8f00-434a-9752-b57582aeadbc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:15.240777 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c00e86-8f00-434a-9752-b57582aeadbc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:15.242383 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c00e86-8f00-434a-9752-b57582aeadbc None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:15.256868 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-d3c00e86-8f00-434a-9752-b57582aeadbc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:57:15.256868 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:15.256868 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:15.256868 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:15.256868 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:15.256868 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:15.256868 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:15.256868 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:15.256868 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:15.256868 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:15.256868 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:15.256868 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:15.256868 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:57:15.256868 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:57:15.256868 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:57:15.256868 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:57:15.258342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:15.258342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:15.258342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:57:15.258342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:57:15.258342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:57:15.258342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:57:15.258342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:15.258342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:15.258342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:15.258342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:15.258342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:15.258342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:15.258342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:15.258342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:15.258342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:15.258342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:15.258342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:15.258342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:15.259429 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:15.259429 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:15.259429 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:57:15.259429 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:57:15.259429 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:57:15.259429 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:57:15.259429 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:57:15.259429 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:57:15.259429 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:57:15.259429 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:15.259429 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 603/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:57:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:15.264427 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4dc6e9-fccb-438e-bf87-1a21c814ad72 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:15.316049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:15.372302 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4dc6e9-fccb-438e-bf87-1a21c814ad72 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:15.412372 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4aac69-537e-402a-b175-de01079ee070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:15.412977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4aac69-537e-402a-b175-de01079ee070 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:15.413254 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4aac69-537e-402a-b175-de01079ee070 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:15.413524 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4aac69-537e-402a-b175-de01079ee070 None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:15.414271 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4aac69-537e-402a-b175-de01079ee070 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:15.414574 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4aac69-537e-402a-b175-de01079ee070 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:15.415994 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4aac69-537e-402a-b175-de01079ee070 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:15.434217 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 624/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Sep 14 01:57:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:15.439992 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c00e86-8f00-434a-9752-b57582aeadbc None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:15.546445 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c00e86-8f00-434a-9752-b57582aeadbc None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:15.583355 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4736305-7a62-4a43-ac06-21aa1f9ad220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:15.583986 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4736305-7a62-4a43-ac06-21aa1f9ad220 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:15.584264 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4736305-7a62-4a43-ac06-21aa1f9ad220 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:15.584540 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4736305-7a62-4a43-ac06-21aa1f9ad220 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:15.596654 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4736305-7a62-4a43-ac06-21aa1f9ad220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:15.598337 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4736305-7a62-4a43-ac06-21aa1f9ad220 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:15.619067 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-b4736305-7a62-4a43-ac06-21aa1f9ad220 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:15.619067 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:15.619067 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:15.619067 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:15.619067 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:15.619067 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:15.619067 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:15.619067 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:15.619067 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:15.619067 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:15.619067 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:15.619067 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:15.619067 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:57:15.619067 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:57:15.619067 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:57:15.619067 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:57:15.620702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:15.620702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:15.620702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:15.620702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:15.620702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:15.620702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:15.620702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:15.620702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:15.620702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:15.620702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:15.620702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:15.620702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:15.620702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:15.620702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:15.620702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:15.620702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:15.620702 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:57:15.621681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:57:15.621681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:15.621681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:15.621681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 604/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:57:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:15.627382 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4aac69-537e-402a-b175-de01079ee070 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:15.748657 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4aac69-537e-402a-b175-de01079ee070 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:15.752342 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:15.786121 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd59fc-829b-4c8b-b8e1-da8763a22309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:15.786727 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd59fc-829b-4c8b-b8e1-da8763a22309 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:15.787005 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd59fc-829b-4c8b-b8e1-da8763a22309 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:15.787278 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd59fc-829b-4c8b-b8e1-da8763a22309 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:15.788445 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd59fc-829b-4c8b-b8e1-da8763a22309 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:15.788795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd59fc-829b-4c8b-b8e1-da8763a22309 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:15.790273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd59fc-829b-4c8b-b8e1-da8763a22309 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:15.802809 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 625/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:15.809059 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4736305-7a62-4a43-ac06-21aa1f9ad220 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:15.818269 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:15.912453 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4736305-7a62-4a43-ac06-21aa1f9ad220 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:15.949079 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b8606-430d-436d-9e42-ecb785f36c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:15.949710 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b8606-430d-436d-9e42-ecb785f36c88 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:15.950003 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b8606-430d-436d-9e42-ecb785f36c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:15.950279 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b8606-430d-436d-9e42-ecb785f36c88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:15.956952 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b8606-430d-436d-9e42-ecb785f36c88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:15.958424 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b8606-430d-436d-9e42-ecb785f36c88 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:15.970744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-483b8606-430d-436d-9e42-ecb785f36c88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:57:15.970744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:15.970744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:15.970744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:15.970744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:15.970744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:15.970744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:15.970744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:15.970744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:15.970744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:15.970744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:15.970744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:15.970744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:57:15.970744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:57:15.970744 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:57:15.971816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:57:15.971816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:15.971816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:15.971816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:15.971816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:15.971816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:15.971816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:15.971816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:15.971816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:15.971816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:15.971816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:15.971816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:15.971816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:15.971816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:15.971816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:15.971816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:15.971816 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:15.972956 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:57:15.972956 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:57:15.972956 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:57:15.972956 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:57:15.972956 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:57:15.972956 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:57:15.972956 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:57:15.972956 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:15.972956 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 605/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:57:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:15.977613 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd59fc-829b-4c8b-b8e1-da8763a22309 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:16.105575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd59fc-829b-4c8b-b8e1-da8763a22309 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:16.145318 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c48ee7-ad94-4fe3-94e5-63d0cef426b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:16.145925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c48ee7-ad94-4fe3-94e5-63d0cef426b1 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:16.146248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c48ee7-ad94-4fe3-94e5-63d0cef426b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:16.146517 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c48ee7-ad94-4fe3-94e5-63d0cef426b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:16.147198 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c48ee7-ad94-4fe3-94e5-63d0cef426b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:16.147507 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c48ee7-ad94-4fe3-94e5-63d0cef426b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:16.148995 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c48ee7-ad94-4fe3-94e5-63d0cef426b1 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:16.166197 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 626/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:57:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:16.167590 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b8606-430d-436d-9e42-ecb785f36c88 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:16.236906 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:16.276048 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b8606-430d-436d-9e42-ecb785f36c88 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:16.316561 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d9920-a9dd-4f60-85f8-f4237924a862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:16.317505 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d9920-a9dd-4f60-85f8-f4237924a862 None admin] REQUEST_METHOD: `PUT` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:16.317757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d9920-a9dd-4f60-85f8-f4237924a862 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:16.318059 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d9920-a9dd-4f60-85f8-f4237924a862 None admin] PATH_INFO: `/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/4da8e712cd9b454fa035f93785775fd8/roles/e962a2a280224f9fbcd0fdc02262993d` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:16.333897 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d9920-a9dd-4f60-85f8-f4237924a862 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e04b16dba1c4de79dc6b1a5e16ed907, user_id=4da8e712cd9b454fa035f93785775fd8, role_id=e962a2a280224f9fbcd0fdc02262993d)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:16.335372 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d9920-a9dd-4f60-85f8-f4237924a862 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:16.348507 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 606/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:57:16 2022] PUT /identity/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/4da8e712cd9b454fa035f93785775fd8/roles/e962a2a280224f9fbcd0fdc02262993d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:57:18.528925 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1d014-c506-413e-9ab8-dfe7478b1494 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:18.529533 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1d014-c506-413e-9ab8-dfe7478b1494 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:18.529533 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1d014-c506-413e-9ab8-dfe7478b1494 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:18.530890 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 627/1233] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:57:18.536944 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fc408-7e38-445d-ab84-181060c6934e None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:18.537482 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fc408-7e38-445d-ab84-181060c6934e None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:18.537482 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fc408-7e38-445d-ab84-181060c6934e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:18.556109 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-2d5fc408-7e38-445d-ab84-181060c6934e None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:57:18.557826 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-2d5fc408-7e38-445d-ab84-181060c6934e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:57:18.629512 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 607/1234] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:57:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:57:18.903863 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1d014-c506-413e-9ab8-dfe7478b1494 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:18.928606 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:19.015057 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1d014-c506-413e-9ab8-dfe7478b1494 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:19.059951 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764ad06-cd14-4274-9259-0ae79ec60d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:19.061021 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764ad06-cd14-4274-9259-0ae79ec60d60 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:19.061514 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764ad06-cd14-4274-9259-0ae79ec60d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:19.062014 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764ad06-cd14-4274-9259-0ae79ec60d60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:19.075683 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764ad06-cd14-4274-9259-0ae79ec60d60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:57:19.081461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-d764ad06-cd14-4274-9259-0ae79ec60d60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:57:19.081461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:19.081461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:19.081461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:19.081461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:19.081461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:19.081461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:19.081461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:19.081461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:19.081461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:19.081461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:19.081461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:19.081461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:57:19.081461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:57:19.081461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:57:19.081461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:57:19.082778 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:57:19.082778 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:57:19.082778 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:57:19.082778 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:57:19.082778 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:19.082778 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:19.082778 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:19.082778 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:19.082778 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:19.082778 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:19.082778 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:19.082778 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:19.082778 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:19.082778 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:19.082778 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:19.082778 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:19.084062 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:19.084062 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:19.084062 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:19.084062 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:19.084062 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:57:19.084062 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:57:19.084062 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:57:19.084062 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:57:19.084062 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:57:19.084062 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:57:19.084062 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:57:19.084062 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:57:19.084062 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 628/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 14 01:57:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:19.089764 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5fc408-7e38-445d-ab84-181060c6934e None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:19.141731 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:19.207472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5fc408-7e38-445d-ab84-181060c6934e None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:19.247326 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72342880-f473-489f-88d9-f1640badc8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:19.250322 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72342880-f473-489f-88d9-f1640badc8df None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:19.250322 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72342880-f473-489f-88d9-f1640badc8df None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:19.250322 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72342880-f473-489f-88d9-f1640badc8df None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:19.250322 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72342880-f473-489f-88d9-f1640badc8df None admin] RBAC: Adding query filter params (name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:19.250322 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72342880-f473-489f-88d9-f1640badc8df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:19.250786 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72342880-f473-489f-88d9-f1640badc8df None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:19.260046 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 608/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 14 01:57:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:19.266258 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764ad06-cd14-4274-9259-0ae79ec60d60 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:19.358512 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:19.376106 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764ad06-cd14-4274-9259-0ae79ec60d60 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:19.416256 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ffdbf-b4a2-45d8-9176-58d6058c3bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:19.416902 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ffdbf-b4a2-45d8-9176-58d6058c3bad None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:19.417218 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ffdbf-b4a2-45d8-9176-58d6058c3bad None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:19.417460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ffdbf-b4a2-45d8-9176-58d6058c3bad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:19.428241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ffdbf-b4a2-45d8-9176-58d6058c3bad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:19.429713 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ffdbf-b4a2-45d8-9176-58d6058c3bad None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:19.444883 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-1e2ffdbf-b4a2-45d8-9176-58d6058c3bad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:19.444883 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:19.444883 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:19.444883 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:19.444883 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:19.444883 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:19.444883 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:19.444883 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:19.444883 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:19.444883 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:19.444883 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:19.444883 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:19.444883 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:57:19.444883 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:57:19.444883 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:57:19.444883 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:57:19.446253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:19.446253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:19.446253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:19.446253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:19.446253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:19.446253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:19.446253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:19.446253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:19.446253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:19.446253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:19.446253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:19.446253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:19.446253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:19.446253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:19.446253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:19.446253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:19.446253 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:57:19.447468 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:57:19.447468 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:19.447468 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:57:19.447468 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 629/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:57:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:19.453038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72342880-f473-489f-88d9-f1640badc8df None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:19.562894 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72342880-f473-489f-88d9-f1640badc8df None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:19.574063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:19.603025 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da433a8f-c31d-4ff0-b4da-8cbe1471bf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:19.603634 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da433a8f-c31d-4ff0-b4da-8cbe1471bf98 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:19.603913 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da433a8f-c31d-4ff0-b4da-8cbe1471bf98 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:19.604211 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da433a8f-c31d-4ff0-b4da-8cbe1471bf98 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:19.604851 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da433a8f-c31d-4ff0-b4da-8cbe1471bf98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:19.605188 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da433a8f-c31d-4ff0-b4da-8cbe1471bf98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:19.606572 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da433a8f-c31d-4ff0-b4da-8cbe1471bf98 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:19.618523 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 609/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:57:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:57:19.625341 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ffdbf-b4a2-45d8-9176-58d6058c3bad None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:19.733910 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ffdbf-b4a2-45d8-9176-58d6058c3bad None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:19.770463 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0cf07c-25f8-415c-85a5-ac4a131c0e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:19.771092 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cf07c-25f8-415c-85a5-ac4a131c0e31 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:19.771368 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cf07c-25f8-415c-85a5-ac4a131c0e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:19.771695 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cf07c-25f8-415c-85a5-ac4a131c0e31 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:19.776098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:19.781864 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0cf07c-25f8-415c-85a5-ac4a131c0e31 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:19.783847 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0cf07c-25f8-415c-85a5-ac4a131c0e31 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:19.796764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-2f0cf07c-25f8-415c-85a5-ac4a131c0e31 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:57:19.796764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:19.796764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:19.796764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:19.796764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:19.796764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:19.796764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:19.796764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:19.796764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:19.796764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:19.796764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:19.796764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:19.796764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:57:19.796764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:57:19.796764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:57:19.796764 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:57:19.798038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:19.798038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:19.798038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:57:19.798038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:57:19.798038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:57:19.798038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:57:19.798038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:19.798038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:19.798038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:19.798038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:19.798038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:19.798038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:19.798038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:19.798038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:19.798038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:19.798038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:19.798038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:19.798038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:19.799146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:19.799146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:19.799146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:57:19.799146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:57:19.799146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:57:19.799146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:57:19.799146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:57:19.799146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:57:19.799146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:57:19.799146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:57:19.799146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 630/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:57:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:19.803812 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da433a8f-c31d-4ff0-b4da-8cbe1471bf98 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:19.914996 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da433a8f-c31d-4ff0-b4da-8cbe1471bf98 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:19.953102 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083bb15c-851b-436b-a592-939eaac1ef2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:19.953736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bb15c-851b-436b-a592-939eaac1ef2e None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:19.954038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bb15c-851b-436b-a592-939eaac1ef2e None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:19.954312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bb15c-851b-436b-a592-939eaac1ef2e None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:19.955149 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083bb15c-851b-436b-a592-939eaac1ef2e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:19.955533 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083bb15c-851b-436b-a592-939eaac1ef2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:19.957088 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083bb15c-851b-436b-a592-939eaac1ef2e None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:19.983983 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 610/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Sep 14 01:57:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:57:19.990053 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0cf07c-25f8-415c-85a5-ac4a131c0e31 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:20.112135 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0cf07c-25f8-415c-85a5-ac4a131c0e31 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:20.149211 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae87a3c-b027-4efc-a6e6-58b7c29978eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:20.149827 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae87a3c-b027-4efc-a6e6-58b7c29978eb None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:20.150144 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae87a3c-b027-4efc-a6e6-58b7c29978eb None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:20.150520 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae87a3c-b027-4efc-a6e6-58b7c29978eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:20.217693 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:20.227006 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae87a3c-b027-4efc-a6e6-58b7c29978eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:20.228466 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae87a3c-b027-4efc-a6e6-58b7c29978eb None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:20.347089 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 631/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:57:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:57:20.353506 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083bb15c-851b-436b-a592-939eaac1ef2e None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:20.385064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:20.469443 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083bb15c-851b-436b-a592-939eaac1ef2e None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:20.507406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd621f9e-dc02-4ce4-a9f6-bab790eb6f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:20.508041 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd621f9e-dc02-4ce4-a9f6-bab790eb6f09 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:20.508318 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd621f9e-dc02-4ce4-a9f6-bab790eb6f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:20.508587 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd621f9e-dc02-4ce4-a9f6-bab790eb6f09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:20.518868 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd621f9e-dc02-4ce4-a9f6-bab790eb6f09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:20.520314 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd621f9e-dc02-4ce4-a9f6-bab790eb6f09 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:20.536117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-cd621f9e-dc02-4ce4-a9f6-bab790eb6f09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:20.536117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:20.536117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:20.536117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:20.536117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:20.536117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:20.536117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:20.536117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:20.536117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:20.536117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:20.536117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:20.536117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:20.536117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:57:20.536117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:57:20.536117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:57:20.536117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:57:20.537184 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:20.537184 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:20.537184 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:20.537184 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:20.537184 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:20.537184 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:20.537184 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:20.537184 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:20.537184 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:20.537184 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:20.537184 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:20.537184 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:20.537184 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:20.537184 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:20.537184 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:20.537184 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:20.537184 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:57:20.538739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:57:20.538739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:20.538739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:20.538739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 611/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:57:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:20.543797 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae87a3c-b027-4efc-a6e6-58b7c29978eb None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:20.643772 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:20.650220 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae87a3c-b027-4efc-a6e6-58b7c29978eb None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:20.687854 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f457040-e631-4ea7-a6cf-2ee58d2369b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:20.688488 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f457040-e631-4ea7-a6cf-2ee58d2369b5 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:20.688770 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f457040-e631-4ea7-a6cf-2ee58d2369b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:20.689044 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f457040-e631-4ea7-a6cf-2ee58d2369b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:20.689676 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f457040-e631-4ea7-a6cf-2ee58d2369b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:20.690011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f457040-e631-4ea7-a6cf-2ee58d2369b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:20.691378 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f457040-e631-4ea7-a6cf-2ee58d2369b5 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:20.703526 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 632/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:20.709401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd621f9e-dc02-4ce4-a9f6-bab790eb6f09 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:20.797590 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:20.820268 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd621f9e-dc02-4ce4-a9f6-bab790eb6f09 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:20.858718 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab19e7-697f-482f-812b-912246b9cfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:20.859511 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab19e7-697f-482f-812b-912246b9cfdc None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:20.859850 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab19e7-697f-482f-812b-912246b9cfdc None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:20.860182 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab19e7-697f-482f-812b-912246b9cfdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:20.867281 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab19e7-697f-482f-812b-912246b9cfdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:20.868717 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab19e7-697f-482f-812b-912246b9cfdc None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:20.880899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-90ab19e7-697f-482f-812b-912246b9cfdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:57:20.880899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:20.880899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:20.880899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:20.880899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:20.880899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:20.880899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:20.880899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:20.880899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:20.880899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:20.880899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:20.880899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:20.880899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:57:20.880899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:57:20.880899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:57:20.882101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:57:20.882101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:20.882101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:20.882101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:20.882101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:20.882101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:20.882101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:20.882101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:20.882101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:20.882101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:20.882101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:20.882101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:20.882101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:20.882101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:20.882101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:20.882101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:20.882101 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:20.883048 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:57:20.883048 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:57:20.883048 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:57:20.883048 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:57:20.883048 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:57:20.883048 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:57:20.883048 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:57:20.883048 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:20.883048 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 612/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:57:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:57:20.889277 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f457040-e631-4ea7-a6cf-2ee58d2369b5 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:21.001371 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f457040-e631-4ea7-a6cf-2ee58d2369b5 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:21.041072 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28595c39-4c97-44a1-b14f-ec2fbb3ebf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:21.041817 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28595c39-4c97-44a1-b14f-ec2fbb3ebf64 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:21.042198 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28595c39-4c97-44a1-b14f-ec2fbb3ebf64 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:21.042540 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28595c39-4c97-44a1-b14f-ec2fbb3ebf64 None admin] PATH_INFO: `/v3/projects` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:21.043223 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28595c39-4c97-44a1-b14f-ec2fbb3ebf64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:21.043528 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28595c39-4c97-44a1-b14f-ec2fbb3ebf64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:21.044972 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28595c39-4c97-44a1-b14f-ec2fbb3ebf64 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:21.062151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 633/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:57:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:21.071097 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab19e7-697f-482f-812b-912246b9cfdc None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:21.187128 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab19e7-697f-482f-812b-912246b9cfdc None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:21.224675 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f6574-04f0-490b-bf34-f17dd851fac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:21.225383 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f6574-04f0-490b-bf34-f17dd851fac9 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:21.225709 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f6574-04f0-490b-bf34-f17dd851fac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:21.225967 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f6574-04f0-490b-bf34-f17dd851fac9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:21.226860 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f6574-04f0-490b-bf34-f17dd851fac9 None admin] RBAC: Adding query filter params (role.id=e962a2a280224f9fbcd0fdc02262993d, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=4da8e712cd9b454fa035f93785775fd8) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:21.227145 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f6574-04f0-490b-bf34-f17dd851fac9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:21.228806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f6574-04f0-490b-bf34-f17dd851fac9 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:21.236326 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 613/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:57:21 2022] GET /identity/v3/role_assignments?user.id=4da8e712cd9b454fa035f93785775fd8&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=e962a2a280224f9fbcd0fdc02262993d => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:23.615381 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7edcd-fd79-41c4-9f7b-18ddb7bbe56c None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:23.615967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7edcd-fd79-41c4-9f7b-18ddb7bbe56c None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:23.615967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7edcd-fd79-41c4-9f7b-18ddb7bbe56c None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:23.617248 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 634/1247] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:57:23.623214 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ece634-e745-4c3e-bdda-89d859a46752 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:23.623620 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ece634-e745-4c3e-bdda-89d859a46752 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:23.623847 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ece634-e745-4c3e-bdda-89d859a46752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:23.630599 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:23.644587 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-61ece634-e745-4c3e-bdda-89d859a46752 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:57:23.645861 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-61ece634-e745-4c3e-bdda-89d859a46752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:57:23.715018 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 614/1248] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:57:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:57:23.983686 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7edcd-fd79-41c4-9f7b-18ddb7bbe56c None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:24.004274 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:24.109378 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7edcd-fd79-41c4-9f7b-18ddb7bbe56c None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:24.146468 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c44f9-5fe3-4616-b5e5-ee6d7c939575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:24.147123 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c44f9-5fe3-4616-b5e5-ee6d7c939575 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:24.147405 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c44f9-5fe3-4616-b5e5-ee6d7c939575 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:24.147680 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c44f9-5fe3-4616-b5e5-ee6d7c939575 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:24.159064 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c44f9-5fe3-4616-b5e5-ee6d7c939575 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:57:24.164078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-887c44f9-5fe3-4616-b5e5-ee6d7c939575 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:57:24.164078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:24.164078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:24.164078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:24.164078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:24.164078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:24.164078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:24.164078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:24.164078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:24.164078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:24.164078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:24.164078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:24.164078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:57:24.164078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:57:24.164078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:57:24.164078 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:57:24.165269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:57:24.165269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:57:24.165269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:57:24.165269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:57:24.165269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:24.165269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:24.165269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:24.165269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:24.165269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:24.165269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:24.165269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:24.165269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:24.165269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:24.165269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:24.165269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:24.165269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:24.166864 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:24.166864 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:24.166864 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:24.166864 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:24.166864 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:57:24.166864 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:57:24.166864 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:57:24.166864 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:57:24.166864 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:57:24.166864 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:57:24.166864 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:57:24.166864 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:57:24.166864 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 635/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:57:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:24.171888 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ece634-e745-4c3e-bdda-89d859a46752 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:24.277697 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ece634-e745-4c3e-bdda-89d859a46752 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:24.308635 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:24.314051 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32747262-d597-413d-b405-06a78515aba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:24.314669 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32747262-d597-413d-b405-06a78515aba6 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:24.314947 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32747262-d597-413d-b405-06a78515aba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:24.315219 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32747262-d597-413d-b405-06a78515aba6 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:24.315890 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32747262-d597-413d-b405-06a78515aba6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:24.316191 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32747262-d597-413d-b405-06a78515aba6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:24.317562 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32747262-d597-413d-b405-06a78515aba6 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:24.326302 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 615/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:57:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:24.332273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c44f9-5fe3-4616-b5e5-ee6d7c939575 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:24.421243 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:24.447607 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c44f9-5fe3-4616-b5e5-ee6d7c939575 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:24.483817 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c493a-c8e0-412b-b384-388d90badc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:24.484462 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c493a-c8e0-412b-b384-388d90badc19 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:24.484740 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c493a-c8e0-412b-b384-388d90badc19 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:24.485011 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c493a-c8e0-412b-b384-388d90badc19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:24.497897 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c493a-c8e0-412b-b384-388d90badc19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:24.499403 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c493a-c8e0-412b-b384-388d90badc19 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:24.515750 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-0e8c493a-c8e0-412b-b384-388d90badc19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:24.515750 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:24.515750 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:24.515750 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:24.515750 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:24.515750 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:24.515750 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:24.515750 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:24.515750 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:24.515750 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:24.515750 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:24.515750 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:24.515750 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:57:24.515750 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:57:24.515750 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:57:24.515750 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:57:24.516852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:24.516852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:24.516852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:24.516852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:24.516852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:24.516852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:24.516852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:24.516852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:24.516852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:24.516852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:24.516852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:24.516852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:24.516852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:24.516852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:24.516852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:24.516852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:24.516852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:57:24.517909 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:57:24.517909 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:24.517909 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:57:24.517909 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 636/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:57:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:24.522743 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32747262-d597-413d-b405-06a78515aba6 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:24.622031 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32747262-d597-413d-b405-06a78515aba6 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:24.657034 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eaa168-aaf5-475e-a3fb-0e49676b89b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:24.657661 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eaa168-aaf5-475e-a3fb-0e49676b89b4 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:24.657906 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eaa168-aaf5-475e-a3fb-0e49676b89b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:24.658207 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eaa168-aaf5-475e-a3fb-0e49676b89b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:24.658852 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eaa168-aaf5-475e-a3fb-0e49676b89b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:24.659154 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eaa168-aaf5-475e-a3fb-0e49676b89b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:24.660523 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eaa168-aaf5-475e-a3fb-0e49676b89b4 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:24.673250 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 616/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:24.678831 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c493a-c8e0-412b-b384-388d90badc19 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:24.778771 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c493a-c8e0-412b-b384-388d90badc19 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:24.803957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:24.813953 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df4c2a-8a07-42ed-b4cf-813d9c9099f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:24.814959 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df4c2a-8a07-42ed-b4cf-813d9c9099f6 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:24.815245 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df4c2a-8a07-42ed-b4cf-813d9c9099f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:24.815522 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df4c2a-8a07-42ed-b4cf-813d9c9099f6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:24.823296 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df4c2a-8a07-42ed-b4cf-813d9c9099f6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:24.824728 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df4c2a-8a07-42ed-b4cf-813d9c9099f6 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:24.835642 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-13df4c2a-8a07-42ed-b4cf-813d9c9099f6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:57:24.835642 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:24.835642 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:24.835642 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:24.835642 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:24.835642 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:24.835642 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:24.835642 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:24.835642 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:24.835642 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:24.835642 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:24.835642 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:24.835642 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:57:24.835642 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:57:24.835642 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:57:24.835642 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:57:24.836767 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:24.836767 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:24.836767 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:57:24.836767 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:57:24.836767 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:57:24.836767 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:57:24.836767 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:24.836767 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:24.836767 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:24.836767 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:24.836767 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:24.836767 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:24.836767 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:24.836767 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:24.836767 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:24.836767 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:24.836767 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:24.836767 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:24.838049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:24.838049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:24.838049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:57:24.838049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:57:24.838049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:57:24.838049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:57:24.838049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:57:24.838049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:57:24.838049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:57:24.838049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:57:24.838049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 637/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:57:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:24.842591 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eaa168-aaf5-475e-a3fb-0e49676b89b4 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:24.897213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:24.940396 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eaa168-aaf5-475e-a3fb-0e49676b89b4 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:24.972791 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b4e5c-44b3-4574-a965-5397f4bea829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:24.973383 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b4e5c-44b3-4574-a965-5397f4bea829 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:24.973663 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b4e5c-44b3-4574-a965-5397f4bea829 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:24.973932 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b4e5c-44b3-4574-a965-5397f4bea829 None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:24.974672 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22b4e5c-44b3-4574-a965-5397f4bea829 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:24.974974 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22b4e5c-44b3-4574-a965-5397f4bea829 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:24.976385 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22b4e5c-44b3-4574-a965-5397f4bea829 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:24.993246 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 617/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Sep 14 01:57:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:24.998998 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df4c2a-8a07-42ed-b4cf-813d9c9099f6 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:25.106038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df4c2a-8a07-42ed-b4cf-813d9c9099f6 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:25.138928 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b2048-6bc6-41a1-8a4a-8ef6ada1277e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:25.139525 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b2048-6bc6-41a1-8a4a-8ef6ada1277e None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:25.139806 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b2048-6bc6-41a1-8a4a-8ef6ada1277e None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:25.140079 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b2048-6bc6-41a1-8a4a-8ef6ada1277e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:25.192307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:25.204954 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b2048-6bc6-41a1-8a4a-8ef6ada1277e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:25.206436 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b2048-6bc6-41a1-8a4a-8ef6ada1277e None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:25.300132 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 638/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:57:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 14 01:57:25.305978 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b4e5c-44b3-4574-a965-5397f4bea829 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:25.403179 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b4e5c-44b3-4574-a965-5397f4bea829 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:25.416401 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:25.441429 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696a764-3a17-4865-abc5-1aef1b81b7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:25.442079 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696a764-3a17-4865-abc5-1aef1b81b7de None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:25.442407 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696a764-3a17-4865-abc5-1aef1b81b7de None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:25.442626 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696a764-3a17-4865-abc5-1aef1b81b7de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:25.454793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696a764-3a17-4865-abc5-1aef1b81b7de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:25.456226 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696a764-3a17-4865-abc5-1aef1b81b7de None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:25.471155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-7696a764-3a17-4865-abc5-1aef1b81b7de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:25.471155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:25.471155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:25.471155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:25.471155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:25.471155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:25.471155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:25.471155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:25.471155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:25.471155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:25.471155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:25.471155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:25.471155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:57:25.471155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:57:25.471155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:57:25.471155 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:57:25.472359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:25.472359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:25.472359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:25.472359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:25.472359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:25.472359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:25.472359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:25.472359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:25.472359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:25.472359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:25.472359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:25.472359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:25.472359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:25.472359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:25.472359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:25.472359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:25.472359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:57:25.473284 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:57:25.473284 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:25.473284 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:25.473284 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 618/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:57:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:25.477956 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b2048-6bc6-41a1-8a4a-8ef6ada1277e None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:25.560889 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:25.572389 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b2048-6bc6-41a1-8a4a-8ef6ada1277e None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:25.605853 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8e72a-274c-4202-ac3c-fa188bd3fb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:25.606473 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8e72a-274c-4202-ac3c-fa188bd3fb37 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:25.606747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8e72a-274c-4202-ac3c-fa188bd3fb37 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:25.607014 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8e72a-274c-4202-ac3c-fa188bd3fb37 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:25.607634 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8e72a-274c-4202-ac3c-fa188bd3fb37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:25.607934 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8e72a-274c-4202-ac3c-fa188bd3fb37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:25.609285 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8e72a-274c-4202-ac3c-fa188bd3fb37 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:25.619944 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 639/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:25.625420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696a764-3a17-4865-abc5-1aef1b81b7de None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:25.722383 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696a764-3a17-4865-abc5-1aef1b81b7de None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:25.754488 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a7bc2-60e1-4ae2-b348-864e8dfb8321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:25.755102 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a7bc2-60e1-4ae2-b348-864e8dfb8321 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:25.755378 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a7bc2-60e1-4ae2-b348-864e8dfb8321 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:25.755646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a7bc2-60e1-4ae2-b348-864e8dfb8321 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:25.762059 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a7bc2-60e1-4ae2-b348-864e8dfb8321 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:25.763490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a7bc2-60e1-4ae2-b348-864e8dfb8321 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:25.774519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-6f7a7bc2-60e1-4ae2-b348-864e8dfb8321 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:57:25.774519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:25.774519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:25.774519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:25.774519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:25.774519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:25.774519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:25.774519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:25.774519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:25.774519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:25.774519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:25.774519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:25.774519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:57:25.774519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:57:25.774519 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:57:25.775606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:57:25.775606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:25.775606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:25.775606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:25.775606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:25.775606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:25.775606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:25.775606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:25.775606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:25.775606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:25.775606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:25.775606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:25.775606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:25.775606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:25.775606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:25.775606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:25.775606 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:25.777074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:57:25.777074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:57:25.777074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:57:25.777074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:57:25.777074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:57:25.777074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:57:25.777074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:57:25.777074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:25.777074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 619/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:57:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:25.787207 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8e72a-274c-4202-ac3c-fa188bd3fb37 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:25.890299 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8e72a-274c-4202-ac3c-fa188bd3fb37 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:25.923138 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820aecc-b133-406f-9fed-7018f6bacaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:25.923740 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820aecc-b133-406f-9fed-7018f6bacaf5 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:25.924019 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820aecc-b133-406f-9fed-7018f6bacaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:25.924287 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820aecc-b133-406f-9fed-7018f6bacaf5 None admin] PATH_INFO: `/v3/projects` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:25.924965 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e820aecc-b133-406f-9fed-7018f6bacaf5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:25.925269 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e820aecc-b133-406f-9fed-7018f6bacaf5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:25.926731 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e820aecc-b133-406f-9fed-7018f6bacaf5 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:25.938902 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 640/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:57:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:25.945193 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a7bc2-60e1-4ae2-b348-864e8dfb8321 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:25.970697 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:26.056232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a7bc2-60e1-4ae2-b348-864e8dfb8321 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:26.096373 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d628b83-f136-46d4-be82-8fb1b3496e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:26.096965 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d628b83-f136-46d4-be82-8fb1b3496e93 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:26.097239 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d628b83-f136-46d4-be82-8fb1b3496e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:26.097504 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d628b83-f136-46d4-be82-8fb1b3496e93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:26.098238 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d628b83-f136-46d4-be82-8fb1b3496e93 None admin] RBAC: Adding query filter params (role.id=aa420c2fb1ab4bb3b05b004a4d89779a, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=4da8e712cd9b454fa035f93785775fd8) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:26.098536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d628b83-f136-46d4-be82-8fb1b3496e93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:26.099953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d628b83-f136-46d4-be82-8fb1b3496e93 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:26.105695 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 620/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:57:25 2022] GET /identity/v3/role_assignments?user.id=4da8e712cd9b454fa035f93785775fd8&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=aa420c2fb1ab4bb3b05b004a4d89779a => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:28.266221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17cac0-61d8-4274-8404-5504e2b02b05 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:28.266774 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17cac0-61d8-4274-8404-5504e2b02b05 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:28.266774 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17cac0-61d8-4274-8404-5504e2b02b05 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:28.268123 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 641/1261] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:57:28.273226 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323f69a-1315-479e-8a0a-5ead17d215b0 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:28.273673 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323f69a-1315-479e-8a0a-5ead17d215b0 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:28.273880 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323f69a-1315-479e-8a0a-5ead17d215b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:28.293000 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-1323f69a-1315-479e-8a0a-5ead17d215b0 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:57:28.294213 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-1323f69a-1315-479e-8a0a-5ead17d215b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:57:28.358231 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 621/1262] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:57:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:57:28.633419 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17cac0-61d8-4274-8404-5504e2b02b05 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:28.644822 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:28.735535 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17cac0-61d8-4274-8404-5504e2b02b05 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:28.767871 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720182b7-aa94-4bf9-824c-fb124393fb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:28.768484 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720182b7-aa94-4bf9-824c-fb124393fb7f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:28.768765 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720182b7-aa94-4bf9-824c-fb124393fb7f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:28.769037 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720182b7-aa94-4bf9-824c-fb124393fb7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:28.779194 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720182b7-aa94-4bf9-824c-fb124393fb7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:57:28.784099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-720182b7-aa94-4bf9-824c-fb124393fb7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:57:28.784099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:28.784099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:28.784099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:28.784099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:28.784099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:28.784099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:28.784099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:28.784099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:28.784099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:28.784099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:28.784099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:28.784099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:57:28.784099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:57:28.784099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:57:28.784099 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:57:28.785273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:57:28.785273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:57:28.785273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:57:28.785273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:57:28.785273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:28.785273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:28.785273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:28.785273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:28.785273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:28.785273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:28.785273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:28.785273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:28.785273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:28.785273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:28.785273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:28.785273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:28.786351 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:28.786351 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:28.786351 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:28.786351 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:28.786351 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:57:28.786351 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:57:28.786351 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:57:28.786351 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:57:28.786351 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:57:28.786351 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:57:28.786351 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:57:28.786351 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:57:28.786351 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 642/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:57:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:28.791366 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323f69a-1315-479e-8a0a-5ead17d215b0 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:28.809573 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:28.888436 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323f69a-1315-479e-8a0a-5ead17d215b0 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:28.921107 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3ff7f-5fa0-4aee-8372-e699be3c6e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:28.921699 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3ff7f-5fa0-4aee-8372-e699be3c6e18 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:28.922005 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3ff7f-5fa0-4aee-8372-e699be3c6e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:28.922280 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3ff7f-5fa0-4aee-8372-e699be3c6e18 None admin] PATH_INFO: `/v3/roles` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:28.922927 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3ff7f-5fa0-4aee-8372-e699be3c6e18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:28.923225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3ff7f-5fa0-4aee-8372-e699be3c6e18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:28.924569 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3ff7f-5fa0-4aee-8372-e699be3c6e18 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:28.932289 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 622/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:57:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:28.937966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720182b7-aa94-4bf9-824c-fb124393fb7f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:29.002600 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:29.037323 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720182b7-aa94-4bf9-824c-fb124393fb7f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:29.083483 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d56eff-cebb-4b2f-970c-84ec8b61db01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:29.084598 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d56eff-cebb-4b2f-970c-84ec8b61db01 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:29.085117 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d56eff-cebb-4b2f-970c-84ec8b61db01 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:29.085626 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d56eff-cebb-4b2f-970c-84ec8b61db01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:29.097323 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d56eff-cebb-4b2f-970c-84ec8b61db01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:29.098837 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d56eff-cebb-4b2f-970c-84ec8b61db01 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:29.114733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-a1d56eff-cebb-4b2f-970c-84ec8b61db01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:29.114733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:29.114733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:29.114733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:29.114733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:29.114733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:29.114733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:29.114733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:29.114733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:29.114733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:29.114733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:29.114733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:29.114733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:57:29.114733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:57:29.114733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:57:29.114733 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:57:29.115933 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:29.115933 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:29.115933 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:29.115933 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:29.115933 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:29.115933 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:29.115933 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:29.115933 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:29.115933 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:29.115933 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:29.115933 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:29.115933 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:29.115933 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:29.115933 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:29.115933 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:29.115933 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:29.115933 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:57:29.117177 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:57:29.117177 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:29.117177 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:57:29.117177 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 643/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:57:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:29.121621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3ff7f-5fa0-4aee-8372-e699be3c6e18 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:29.200990 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:29.219949 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3ff7f-5fa0-4aee-8372-e699be3c6e18 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:29.251378 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbeff65-d4b2-4772-92b2-1fb15683ade3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:29.251958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbeff65-d4b2-4772-92b2-1fb15683ade3 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:29.252234 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbeff65-d4b2-4772-92b2-1fb15683ade3 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:29.252501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbeff65-d4b2-4772-92b2-1fb15683ade3 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:29.253129 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbeff65-d4b2-4772-92b2-1fb15683ade3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:29.253434 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbeff65-d4b2-4772-92b2-1fb15683ade3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:29.255066 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbeff65-d4b2-4772-92b2-1fb15683ade3 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:29.266779 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 623/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:57:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:29.272330 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d56eff-cebb-4b2f-970c-84ec8b61db01 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:29.366682 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d56eff-cebb-4b2f-970c-84ec8b61db01 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:29.386229 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:29.400576 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e8b14-b31e-4e1e-a0fa-9b697bd99365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:29.401744 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e8b14-b31e-4e1e-a0fa-9b697bd99365 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:29.402050 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e8b14-b31e-4e1e-a0fa-9b697bd99365 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:29.402326 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e8b14-b31e-4e1e-a0fa-9b697bd99365 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:29.411509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e8b14-b31e-4e1e-a0fa-9b697bd99365 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:29.412955 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e8b14-b31e-4e1e-a0fa-9b697bd99365 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:29.425364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-716e8b14-b31e-4e1e-a0fa-9b697bd99365 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:57:29.425364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:29.425364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:29.425364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:29.425364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:29.425364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:29.425364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:29.425364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:29.425364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:29.425364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:29.425364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:29.425364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:29.425364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:57:29.425364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:57:29.425364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:57:29.425364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:57:29.426509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:29.426509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:29.426509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:57:29.426509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:57:29.426509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:57:29.426509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:57:29.426509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:29.426509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:29.426509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:29.426509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:29.426509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:29.426509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:29.426509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:29.426509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:29.426509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:29.426509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:29.426509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:29.426509 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:29.427697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:29.427697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:29.427697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:57:29.427697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:57:29.427697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:57:29.427697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:57:29.427697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:57:29.427697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:57:29.427697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:57:29.427697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:57:29.427697 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 644/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:57:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:29.432806 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbeff65-d4b2-4772-92b2-1fb15683ade3 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:29.530387 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbeff65-d4b2-4772-92b2-1fb15683ade3 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:29.562890 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55b760-43ad-4ba7-b05b-c1cbb3d905c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:29.563481 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55b760-43ad-4ba7-b05b-c1cbb3d905c4 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:29.563757 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55b760-43ad-4ba7-b05b-c1cbb3d905c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:29.564026 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55b760-43ad-4ba7-b05b-c1cbb3d905c4 None admin] PATH_INFO: `/v3/users` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:29.564734 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55b760-43ad-4ba7-b05b-c1cbb3d905c4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:29.565032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55b760-43ad-4ba7-b05b-c1cbb3d905c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:29.566475 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55b760-43ad-4ba7-b05b-c1cbb3d905c4 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:29.575324 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:29.583935 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 624/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Sep 14 01:57:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:29.590268 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e8b14-b31e-4e1e-a0fa-9b697bd99365 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:29.686209 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e8b14-b31e-4e1e-a0fa-9b697bd99365 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:29.717923 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d590ea2-7034-400f-958b-13fa819fb043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:29.718564 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d590ea2-7034-400f-958b-13fa819fb043 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:29.718840 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d590ea2-7034-400f-958b-13fa819fb043 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:29.719112 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d590ea2-7034-400f-958b-13fa819fb043 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:29.729705 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d590ea2-7034-400f-958b-13fa819fb043 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:29.731167 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d590ea2-7034-400f-958b-13fa819fb043 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:29.747168 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-6d590ea2-7034-400f-958b-13fa819fb043 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:29.747168 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:29.747168 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:29.747168 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:29.747168 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:29.747168 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:29.747168 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:29.747168 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:29.747168 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:29.747168 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:29.747168 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:29.747168 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:29.747168 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:57:29.747168 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:57:29.747168 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:57:29.747168 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:57:29.748161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:29.748161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:29.748161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:29.748161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:29.748161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:29.748161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:29.748161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:29.748161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:29.748161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:29.748161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:29.748161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:29.748161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:29.748161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:29.748161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:29.748161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:29.748161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:29.748161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:57:29.749120 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:57:29.749120 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:29.749120 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:57:29.749120 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 645/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:57:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:29.754784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55b760-43ad-4ba7-b05b-c1cbb3d905c4 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:29.850078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55b760-43ad-4ba7-b05b-c1cbb3d905c4 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:29.882420 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864003f-4ef3-48c5-b726-78dad2f4b964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:29.883010 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864003f-4ef3-48c5-b726-78dad2f4b964 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:29.883288 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864003f-4ef3-48c5-b726-78dad2f4b964 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:29.883557 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864003f-4ef3-48c5-b726-78dad2f4b964 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:29.884188 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864003f-4ef3-48c5-b726-78dad2f4b964 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:29.884487 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864003f-4ef3-48c5-b726-78dad2f4b964 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:29.886413 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864003f-4ef3-48c5-b726-78dad2f4b964 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:29.897221 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 625/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:57:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:29.903237 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d590ea2-7034-400f-958b-13fa819fb043 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:29.933350 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:29.998257 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d590ea2-7034-400f-958b-13fa819fb043 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:30.033158 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727dd50-a927-4f76-a8b6-dd769a9f3455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:30.033858 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727dd50-a927-4f76-a8b6-dd769a9f3455 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:30.034221 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727dd50-a927-4f76-a8b6-dd769a9f3455 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:30.034521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727dd50-a927-4f76-a8b6-dd769a9f3455 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:30.041734 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727dd50-a927-4f76-a8b6-dd769a9f3455 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:30.042922 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727dd50-a927-4f76-a8b6-dd769a9f3455 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:30.056651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-6727dd50-a927-4f76-a8b6-dd769a9f3455 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:57:30.056651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:30.056651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:30.056651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:30.056651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:30.056651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:30.056651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:30.056651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:30.056651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:30.056651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:30.056651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:30.056651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:30.056651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:57:30.056651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:57:30.056651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:57:30.057780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:57:30.057780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:30.057780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:30.057780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:30.057780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:30.057780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:30.057780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:30.057780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:30.057780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:30.057780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:30.057780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:30.057780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:30.057780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:30.057780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:30.057780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:30.057780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:30.057780 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:30.058957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:57:30.058957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:57:30.058957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:57:30.058957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:57:30.058957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:57:30.058957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:57:30.058957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:57:30.058957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:57:30.058957 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 646/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:57:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:30.065891 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864003f-4ef3-48c5-b726-78dad2f4b964 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:30.124632 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:30.165501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864003f-4ef3-48c5-b726-78dad2f4b964 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:30.198359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282edc2-d2f8-4ee7-9aee-2cc92fe29037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:30.198952 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282edc2-d2f8-4ee7-9aee-2cc92fe29037 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:30.199232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282edc2-d2f8-4ee7-9aee-2cc92fe29037 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:30.199504 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282edc2-d2f8-4ee7-9aee-2cc92fe29037 None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:30.200185 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282edc2-d2f8-4ee7-9aee-2cc92fe29037 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:30.200490 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282edc2-d2f8-4ee7-9aee-2cc92fe29037 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:30.201910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282edc2-d2f8-4ee7-9aee-2cc92fe29037 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:30.213426 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 626/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:57:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:30.219415 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727dd50-a927-4f76-a8b6-dd769a9f3455 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:30.309284 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:30.317896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727dd50-a927-4f76-a8b6-dd769a9f3455 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:30.350035 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7db4a0-a31b-4080-89f7-49830c712b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:30.350672 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7db4a0-a31b-4080-89f7-49830c712b07 None admin] REQUEST_METHOD: `PUT` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:30.350951 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7db4a0-a31b-4080-89f7-49830c712b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:30.351220 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7db4a0-a31b-4080-89f7-49830c712b07 None admin] PATH_INFO: `/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/4da8e712cd9b454fa035f93785775fd8/roles/aa420c2fb1ab4bb3b05b004a4d89779a` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:30.353091 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7db4a0-a31b-4080-89f7-49830c712b07 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e04b16dba1c4de79dc6b1a5e16ed907, user_id=4da8e712cd9b454fa035f93785775fd8, role_id=aa420c2fb1ab4bb3b05b004a4d89779a)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:30.354622 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7db4a0-a31b-4080-89f7-49830c712b07 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:30.366280 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 647/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 14 01:57:30 2022] PUT /identity/v3/projects/9e04b16dba1c4de79dc6b1a5e16ed907/users/4da8e712cd9b454fa035f93785775fd8/roles/aa420c2fb1ab4bb3b05b004a4d89779a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:57:32.476915 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6d938-b891-408e-8e77-2fc7b183a4e3 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:32.477428 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6d938-b891-408e-8e77-2fc7b183a4e3 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:32.477428 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6d938-b891-408e-8e77-2fc7b183a4e3 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:32.479403 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 627/1274] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:57:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:57:32.484621 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3a23d-b0d1-4f49-974c-3aa1d9838d4b None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:32.485019 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3a23d-b0d1-4f49-974c-3aa1d9838d4b None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:32.485162 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3a23d-b0d1-4f49-974c-3aa1d9838d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:32.502851 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-2be3a23d-b0d1-4f49-974c-3aa1d9838d4b None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:57:32.504278 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-2be3a23d-b0d1-4f49-974c-3aa1d9838d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:57:32.568682 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 648/1275] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:57:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:57:32.839696 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6d938-b891-408e-8e77-2fc7b183a4e3 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:32.941689 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6d938-b891-408e-8e77-2fc7b183a4e3 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:32.958831 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:32.974919 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd750f88-ddc0-405a-b23c-cde2c3039df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:32.975529 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd750f88-ddc0-405a-b23c-cde2c3039df2 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:32.975801 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd750f88-ddc0-405a-b23c-cde2c3039df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:32.976064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd750f88-ddc0-405a-b23c-cde2c3039df2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:32.986207 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd750f88-ddc0-405a-b23c-cde2c3039df2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:57:32.990882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-fd750f88-ddc0-405a-b23c-cde2c3039df2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:57:32.990882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:32.990882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:32.990882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:32.990882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:32.990882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:32.990882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:32.990882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:32.990882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:32.990882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:32.990882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:32.990882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:32.990882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:57:32.990882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:57:32.990882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:57:32.990882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:57:32.992078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:57:32.992078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:57:32.992078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:57:32.992078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:57:32.992078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:32.992078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:32.992078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:32.992078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:32.992078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:32.992078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:32.992078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:32.992078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:32.992078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:32.992078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:32.992078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:32.992078 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:32.993135 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:32.993135 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:32.993135 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:32.993135 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:32.993135 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:57:32.993135 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:57:32.993135 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:57:32.993135 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:57:32.993135 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:57:32.993135 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:57:32.993135 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:57:32.993135 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:32.993135 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 628/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 14 01:57:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:32.998015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3a23d-b0d1-4f49-974c-3aa1d9838d4b None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:33.084791 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:33.106734 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3a23d-b0d1-4f49-974c-3aa1d9838d4b None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:33.140249 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b944133-83a2-49a8-8d5a-d65bf45b319f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:33.140841 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b944133-83a2-49a8-8d5a-d65bf45b319f None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:33.141120 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b944133-83a2-49a8-8d5a-d65bf45b319f None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:33.141390 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b944133-83a2-49a8-8d5a-d65bf45b319f None admin] PATH_INFO: `/v3/roles` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:33.142062 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b944133-83a2-49a8-8d5a-d65bf45b319f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:33.142363 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b944133-83a2-49a8-8d5a-d65bf45b319f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:33.143717 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b944133-83a2-49a8-8d5a-d65bf45b319f None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:33.151994 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 649/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 14 01:57:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:33.157745 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd750f88-ddc0-405a-b23c-cde2c3039df2 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:33.252176 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd750f88-ddc0-405a-b23c-cde2c3039df2 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:33.284308 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78104567-6489-4d73-81db-b518198699a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:33.284912 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78104567-6489-4d73-81db-b518198699a6 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:33.285186 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78104567-6489-4d73-81db-b518198699a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:33.285451 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78104567-6489-4d73-81db-b518198699a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:33.296105 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78104567-6489-4d73-81db-b518198699a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:33.297536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78104567-6489-4d73-81db-b518198699a6 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:33.312100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-78104567-6489-4d73-81db-b518198699a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:33.312100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:33.312100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:33.312100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:33.312100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:33.312100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:33.312100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:33.312100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:33.312100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:33.312100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:33.312100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:33.312100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:33.312100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:57:33.312100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:57:33.312100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:57:33.312100 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:57:33.313255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:33.313255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:33.313255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:33.313255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:33.313255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:33.313255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:33.313255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:33.313255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:33.313255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:33.313255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:33.313255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:33.313255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:33.313255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:33.313255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:33.313255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:33.313255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:33.313255 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:57:33.314456 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:57:33.314456 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:33.314456 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:33.314456 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 629/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:57:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:33.318888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b944133-83a2-49a8-8d5a-d65bf45b319f None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:33.414222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b944133-83a2-49a8-8d5a-d65bf45b319f None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:33.445902 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b664f29-e3a7-47a7-b58c-9f9e6aa33c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:33.446518 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b664f29-e3a7-47a7-b58c-9f9e6aa33c17 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:33.446792 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b664f29-e3a7-47a7-b58c-9f9e6aa33c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:33.447057 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b664f29-e3a7-47a7-b58c-9f9e6aa33c17 None admin] PATH_INFO: `/v3/domains` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:33.447678 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b664f29-e3a7-47a7-b58c-9f9e6aa33c17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:33.447972 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b664f29-e3a7-47a7-b58c-9f9e6aa33c17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:33.449318 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b664f29-e3a7-47a7-b58c-9f9e6aa33c17 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:33.460510 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 650/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:57:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:33.466808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78104567-6489-4d73-81db-b518198699a6 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:33.496740 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:33.563990 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78104567-6489-4d73-81db-b518198699a6 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:33.596031 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec6f82-f508-4cbc-9ad0-5688060f37e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:33.596648 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec6f82-f508-4cbc-9ad0-5688060f37e4 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:33.596922 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec6f82-f508-4cbc-9ad0-5688060f37e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:33.597190 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec6f82-f508-4cbc-9ad0-5688060f37e4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:33.604851 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec6f82-f508-4cbc-9ad0-5688060f37e4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:33.606309 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec6f82-f508-4cbc-9ad0-5688060f37e4 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:33.617057 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-82ec6f82-f508-4cbc-9ad0-5688060f37e4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:57:33.617057 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:33.617057 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:33.617057 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:33.617057 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:33.617057 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:33.617057 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:33.617057 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:33.617057 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:33.617057 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:33.617057 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:33.617057 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:33.617057 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:57:33.617057 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:57:33.617057 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:57:33.617057 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:57:33.618243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:33.618243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:33.618243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:57:33.618243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:57:33.618243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:57:33.618243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:57:33.618243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:33.618243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:33.618243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:33.618243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:33.618243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:33.618243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:33.618243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:33.618243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:33.618243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:33.618243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:33.618243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:33.618243 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:33.620200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:33.620200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:33.620200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:57:33.620200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:57:33.620200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:57:33.620200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:57:33.620200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:57:33.620200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:57:33.620200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:57:33.620200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:33.620200 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 630/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:57:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:33.624438 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b664f29-e3a7-47a7-b58c-9f9e6aa33c17 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:33.628104 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:33.720279 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b664f29-e3a7-47a7-b58c-9f9e6aa33c17 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:33.753429 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd4bef-a265-4ad6-b437-89683b9dda1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:33.754319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd4bef-a265-4ad6-b437-89683b9dda1c None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:33.754605 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd4bef-a265-4ad6-b437-89683b9dda1c None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:33.754875 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd4bef-a265-4ad6-b437-89683b9dda1c None admin] PATH_INFO: `/v3/users` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:33.755586 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd4bef-a265-4ad6-b437-89683b9dda1c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:33.755888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd4bef-a265-4ad6-b437-89683b9dda1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:33.757307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd4bef-a265-4ad6-b437-89683b9dda1c None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:33.773563 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 651/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Sep 14 01:57:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:33.779237 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec6f82-f508-4cbc-9ad0-5688060f37e4 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:33.865638 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:33.876008 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec6f82-f508-4cbc-9ad0-5688060f37e4 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:33.907426 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf107bf-4c48-4822-8828-da32305f8d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:33.908004 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf107bf-4c48-4822-8828-da32305f8d9f None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:33.908276 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf107bf-4c48-4822-8828-da32305f8d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:33.908543 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf107bf-4c48-4822-8828-da32305f8d9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:33.972050 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf107bf-4c48-4822-8828-da32305f8d9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:33.973413 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf107bf-4c48-4822-8828-da32305f8d9f None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:34.058399 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:34.068275 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 631/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 14 01:57:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:57:34.074668 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd4bef-a265-4ad6-b437-89683b9dda1c None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:34.132835 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:34.169862 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd4bef-a265-4ad6-b437-89683b9dda1c None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:34.202949 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306141d-8424-41dd-b201-a6393fbfe9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:34.203560 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306141d-8424-41dd-b201-a6393fbfe9a5 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:34.203836 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306141d-8424-41dd-b201-a6393fbfe9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:34.204106 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306141d-8424-41dd-b201-a6393fbfe9a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:34.213685 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306141d-8424-41dd-b201-a6393fbfe9a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:34.215161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6306141d-8424-41dd-b201-a6393fbfe9a5 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:34.229183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-6306141d-8424-41dd-b201-a6393fbfe9a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:34.229183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:34.229183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:34.229183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:34.229183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:34.229183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:34.229183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:34.229183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:34.229183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:34.229183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:34.229183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:34.229183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:34.229183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:57:34.229183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:57:34.229183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:57:34.229183 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:57:34.230353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:34.230353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:34.230353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:34.230353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:34.230353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:34.230353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:34.230353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:34.230353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:34.230353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:34.230353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:34.230353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:34.230353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:34.230353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:34.230353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:34.230353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:34.230353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:34.230353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:57:34.231434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:57:34.231434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:57:34.231434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:57:34.231434 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 652/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 14 01:57:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:34.235687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf107bf-4c48-4822-8828-da32305f8d9f None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:34.332734 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf107bf-4c48-4822-8828-da32305f8d9f None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:34.365086 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe3f19-b6cf-4593-bccc-52fcb73b3ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:34.365678 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe3f19-b6cf-4593-bccc-52fcb73b3ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:34.365953 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe3f19-b6cf-4593-bccc-52fcb73b3ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:34.366250 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe3f19-b6cf-4593-bccc-52fcb73b3ca8 None admin] PATH_INFO: `/v3/domains` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:34.366875 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe3f19-b6cf-4593-bccc-52fcb73b3ca8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:34.367170 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe3f19-b6cf-4593-bccc-52fcb73b3ca8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:34.368516 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe3f19-b6cf-4593-bccc-52fcb73b3ca8 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:34.380059 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 632/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 14 01:57:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:34.385698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306141d-8424-41dd-b201-a6393fbfe9a5 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:34.480790 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6306141d-8424-41dd-b201-a6393fbfe9a5 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:34.493632 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:34.513930 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec029a8f-a3cd-4f1d-9c82-c5ea33c6b0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:34.514576 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec029a8f-a3cd-4f1d-9c82-c5ea33c6b0fe None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:34.514860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec029a8f-a3cd-4f1d-9c82-c5ea33c6b0fe None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:34.515135 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec029a8f-a3cd-4f1d-9c82-c5ea33c6b0fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:34.521539 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec029a8f-a3cd-4f1d-9c82-c5ea33c6b0fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:34.523000 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec029a8f-a3cd-4f1d-9c82-c5ea33c6b0fe None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:34.534751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application [None req-ec029a8f-a3cd-4f1d-9c82-c5ea33c6b0fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:57:34.534751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:34.534751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:34.534751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:34.534751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:34.534751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:34.534751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:34.534751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:34.534751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:34.534751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:34.534751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:34.534751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:34.534751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:57:34.534751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:57:34.534751 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:57:34.535815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:57:34.535815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:34.535815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:34.535815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:34.535815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:34.535815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:34.535815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:34.535815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:34.535815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:34.535815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:34.535815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:34.535815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:34.535815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:34.535815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:34.535815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:34.535815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:34.535815 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:34.536803 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:57:34.536803 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:57:34.536803 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:57:34.536803 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:57:34.536803 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:57:34.536803 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:57:34.536803 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:57:34.536803 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: ERROR keystone.server.flask.application Sep 14 01:57:34.536803 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 653/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 14 01:57:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:34.542407 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe3f19-b6cf-4593-bccc-52fcb73b3ca8 None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:34.590899 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:34.638886 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe3f19-b6cf-4593-bccc-52fcb73b3ca8 None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:34.670777 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afd95e-ee7a-4833-853a-5aebdf1648bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:34.671366 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afd95e-ee7a-4833-853a-5aebdf1648bc None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:34.671639 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afd95e-ee7a-4833-853a-5aebdf1648bc None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:34.671906 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afd95e-ee7a-4833-853a-5aebdf1648bc None admin] PATH_INFO: `/v3/projects` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:34.672583 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7afd95e-ee7a-4833-853a-5aebdf1648bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:34.672882 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7afd95e-ee7a-4833-853a-5aebdf1648bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:34.674319 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7afd95e-ee7a-4833-853a-5aebdf1648bc None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:34.686342 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 633/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 14 01:57:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:34.692344 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec029a8f-a3cd-4f1d-9c82-c5ea33c6b0fe None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:34.792318 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec029a8f-a3cd-4f1d-9c82-c5ea33c6b0fe None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:34.823943 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a547c3d-c6e2-4ea4-bff3-cb70b517d6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:34.824529 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a547c3d-c6e2-4ea4-bff3-cb70b517d6be None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:34.824807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a547c3d-c6e2-4ea4-bff3-cb70b517d6be None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:34.825075 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a547c3d-c6e2-4ea4-bff3-cb70b517d6be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:34.825774 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a547c3d-c6e2-4ea4-bff3-cb70b517d6be None admin] RBAC: Adding query filter params (role.id=aa420c2fb1ab4bb3b05b004a4d89779a, scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907, user.id=4da8e712cd9b454fa035f93785775fd8) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:34.826109 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a547c3d-c6e2-4ea4-bff3-cb70b517d6be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:34.827521 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a547c3d-c6e2-4ea4-bff3-cb70b517d6be None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:34.833027 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 654/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 14 01:57:34 2022] GET /identity/v3/role_assignments?user.id=4da8e712cd9b454fa035f93785775fd8&scope.project.id=9e04b16dba1c4de79dc6b1a5e16ed907&role.id=aa420c2fb1ab4bb3b05b004a4d89779a => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:36.970509 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470c406-b5b4-4453-85fb-df63acccb85c None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:36.971068 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470c406-b5b4-4453-85fb-df63acccb85c None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:36.971068 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470c406-b5b4-4453-85fb-df63acccb85c None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:36.972394 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 634/1288] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:57:36.977589 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46591e8d-3952-4552-bc06-dbc1b5fb2a51 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:36.977966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46591e8d-3952-4552-bc06-dbc1b5fb2a51 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:36.978175 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46591e8d-3952-4552-bc06-dbc1b5fb2a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:36.996435 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-46591e8d-3952-4552-bc06-dbc1b5fb2a51 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:57:36.997602 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-46591e8d-3952-4552-bc06-dbc1b5fb2a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:57:37.027917 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:37.071724 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 655/1289] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:57:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:57:37.340494 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470c406-b5b4-4453-85fb-df63acccb85c None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:37.437504 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470c406-b5b4-4453-85fb-df63acccb85c None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:37.448093 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:37.470900 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c621da-9ab6-4f56-88a6-620dfb6ec8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:37.471515 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c621da-9ab6-4f56-88a6-620dfb6ec8da None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:37.471788 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c621da-9ab6-4f56-88a6-620dfb6ec8da None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:37.472054 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c621da-9ab6-4f56-88a6-620dfb6ec8da None admin] PATH_INFO: `/v3/services/placement` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:37.472619 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c621da-9ab6-4f56-88a6-620dfb6ec8da None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:37.473959 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c621da-9ab6-4f56-88a6-620dfb6ec8da None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:37.482498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-88c621da-9ab6-4f56-88a6-620dfb6ec8da None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 01:57:37.482498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:37.482498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:57:37.482498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:57:37.482498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:57:37.482498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:57:37.482498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:57:37.482498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:57:37.482498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 01:57:37.482498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:37.482498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:57:37.482498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:37.482498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:37.482498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:37.482498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:37.482498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:37.482498 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:37.483711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:37.483711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:37.483711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:37.483711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:37.483711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:37.483711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:37.483711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:57:37.483711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:57:37.483711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:57:37.483711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:57:37.483711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:37.483711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:37.483711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:37.483711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:37.483711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:37.483711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:37.484613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:37.484613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:37.484613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:37.484613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:37.484613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:37.484613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:37.484613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:37.484613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:37.484613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:37.484613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:37.484613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:57:37.484613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:57:37.484613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 01:57:37.484613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:37.484613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 635/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Sep 14 01:57:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:37.489526 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46591e8d-3952-4552-bc06-dbc1b5fb2a51 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:37.583969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46591e8d-3952-4552-bc06-dbc1b5fb2a51 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:37.616492 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af63c35-8c93-4dba-8e41-1a9d24016d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:37.617027 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af63c35-8c93-4dba-8e41-1a9d24016d38 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:37.617306 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af63c35-8c93-4dba-8e41-1a9d24016d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:37.617575 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af63c35-8c93-4dba-8e41-1a9d24016d38 None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:37.618215 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af63c35-8c93-4dba-8e41-1a9d24016d38 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:37.618517 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af63c35-8c93-4dba-8e41-1a9d24016d38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:37.619896 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af63c35-8c93-4dba-8e41-1a9d24016d38 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:37.624468 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 656/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Sep 14 01:57:37 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:37.630370 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c621da-9ab6-4f56-88a6-620dfb6ec8da None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:37.725256 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c621da-9ab6-4f56-88a6-620dfb6ec8da None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:37.757004 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5831f0-29a1-47f1-a985-561c85c22a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:37.757587 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5831f0-29a1-47f1-a985-561c85c22a94 None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:37.757861 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5831f0-29a1-47f1-a985-561c85c22a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:37.758157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5831f0-29a1-47f1-a985-561c85c22a94 None admin] PATH_INFO: `/v3/services` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:37.758763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5831f0-29a1-47f1-a985-561c85c22a94 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:37.759059 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5831f0-29a1-47f1-a985-561c85c22a94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:37.760517 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5831f0-29a1-47f1-a985-561c85c22a94 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:37.764675 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 636/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Sep 14 01:57:37 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:39.873552 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c421a36-689f-4066-80f1-25ccb4182255 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:39.874108 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c421a36-689f-4066-80f1-25ccb4182255 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:39.874206 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c421a36-689f-4066-80f1-25ccb4182255 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:39.875454 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 657/1293] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:57:39.880936 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cdad0-05fb-4d8e-aa1e-72378417f7b5 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:39.881357 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cdad0-05fb-4d8e-aa1e-72378417f7b5 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:39.881473 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cdad0-05fb-4d8e-aa1e-72378417f7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:39.899367 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-5e2cdad0-05fb-4d8e-aa1e-72378417f7b5 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:57:39.900544 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-5e2cdad0-05fb-4d8e-aa1e-72378417f7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:57:39.929579 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:39.966742 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 637/1294] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:57:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:57:40.232320 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c421a36-689f-4066-80f1-25ccb4182255 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:40.263460 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:40.331021 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c421a36-689f-4066-80f1-25ccb4182255 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:40.363136 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b66148-1d77-4266-b0b0-1a1f199aeadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:40.363728 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b66148-1d77-4266-b0b0-1a1f199aeadd None admin] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:40.364007 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b66148-1d77-4266-b0b0-1a1f199aeadd None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:40.364277 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b66148-1d77-4266-b0b0-1a1f199aeadd None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:40.364910 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b66148-1d77-4266-b0b0-1a1f199aeadd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:40.366291 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b66148-1d77-4266-b0b0-1a1f199aeadd None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:40.380487 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 658/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 14 01:57:40 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:57:42.510655 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef1795-a0d4-425c-9fe5-dd87c3405967 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:42.511181 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef1795-a0d4-425c-9fe5-dd87c3405967 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:42.511181 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef1795-a0d4-425c-9fe5-dd87c3405967 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:42.512527 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 638/1296] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:57:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:57:42.519565 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa74a0-1949-4b70-a96f-00860a001a26 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:42.519994 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa74a0-1949-4b70-a96f-00860a001a26 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:42.520109 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa74a0-1949-4b70-a96f-00860a001a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:42.537902 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-61aa74a0-1949-4b70-a96f-00860a001a26 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:57:42.539112 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-61aa74a0-1949-4b70-a96f-00860a001a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:57:42.608545 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 659/1297] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:57:42 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:57:42.873024 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef1795-a0d4-425c-9fe5-dd87c3405967 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:42.972743 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef1795-a0d4-425c-9fe5-dd87c3405967 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:43.005867 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f368f-5181-4025-bd4e-cb197a2245fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:43.006509 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f368f-5181-4025-bd4e-cb197a2245fe None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:43.006786 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f368f-5181-4025-bd4e-cb197a2245fe None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:43.007059 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f368f-5181-4025-bd4e-cb197a2245fe None admin] PATH_INFO: `/v3/services/placement` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:43.007628 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f368f-5181-4025-bd4e-cb197a2245fe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:43.008975 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f368f-5181-4025-bd4e-cb197a2245fe None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:43.017501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-e80f368f-5181-4025-bd4e-cb197a2245fe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 01:57:43.017501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:43.017501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:57:43.017501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:57:43.017501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:57:43.017501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:57:43.017501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:57:43.017501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:57:43.017501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 01:57:43.017501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:43.017501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:57:43.017501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:43.017501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:43.017501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:43.017501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:43.017501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:43.017501 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:43.018776 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:43.018776 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:43.018776 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:43.018776 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:43.018776 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:43.018776 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:43.018776 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:57:43.018776 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:57:43.018776 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:57:43.018776 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:57:43.018776 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:43.018776 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:43.018776 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:43.018776 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:43.018776 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:43.018776 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:43.019849 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:43.019849 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:43.019849 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:43.019849 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:43.019849 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:43.019849 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:43.019849 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:43.019849 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:43.019849 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:43.019849 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:43.019849 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:57:43.019849 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:57:43.019849 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 01:57:43.019849 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:43.019849 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 639/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Sep 14 01:57:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:43.028191 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa74a0-1949-4b70-a96f-00860a001a26 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:43.049260 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:43.137569 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa74a0-1949-4b70-a96f-00860a001a26 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:43.171146 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7efb4-c358-4244-8f3e-3e646d9735c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:43.171740 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7efb4-c358-4244-8f3e-3e646d9735c9 None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:43.172017 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7efb4-c358-4244-8f3e-3e646d9735c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:43.172286 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7efb4-c358-4244-8f3e-3e646d9735c9 None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:43.172902 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7efb4-c358-4244-8f3e-3e646d9735c9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:43.173199 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7efb4-c358-4244-8f3e-3e646d9735c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:43.174597 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7efb4-c358-4244-8f3e-3e646d9735c9 None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:43.179456 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 660/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Sep 14 01:57:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:43.185640 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f368f-5181-4025-bd4e-cb197a2245fe None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:43.215942 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:43.281966 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f368f-5181-4025-bd4e-cb197a2245fe None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:43.313657 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6969c6-7401-480b-b7fc-f57e11e36a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:43.314259 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6969c6-7401-480b-b7fc-f57e11e36a2f None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:43.314533 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6969c6-7401-480b-b7fc-f57e11e36a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:43.314800 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6969c6-7401-480b-b7fc-f57e11e36a2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:43.315478 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6969c6-7401-480b-b7fc-f57e11e36a2f None admin] RBAC: Adding query filter params (interface=public, service_id=cc08a4b41463443db1943401c863cfd3, region_id=RegionOne) {{(pid=81383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:43.315775 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6969c6-7401-480b-b7fc-f57e11e36a2f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:43.317114 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6969c6-7401-480b-b7fc-f57e11e36a2f None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:43.321841 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 640/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Sep 14 01:57:43 2022] GET /identity/v3/endpoints?service_id=cc08a4b41463443db1943401c863cfd3&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:43.327931 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7efb4-c358-4244-8f3e-3e646d9735c9 None admin] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:43.406710 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:43.424385 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7efb4-c358-4244-8f3e-3e646d9735c9 None admin] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:43.456680 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb40635-0d5d-4134-9fc2-14c00290e05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:43.457273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb40635-0d5d-4134-9fc2-14c00290e05a None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:43.457550 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb40635-0d5d-4134-9fc2-14c00290e05a None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:43.457820 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb40635-0d5d-4134-9fc2-14c00290e05a None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:43.458440 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb40635-0d5d-4134-9fc2-14c00290e05a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:43.459795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb40635-0d5d-4134-9fc2-14c00290e05a None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:43.466837 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 661/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Sep 14 01:57:43 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:57:45.591594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a25cf9-3017-46bf-b460-95c3ffd3fb38 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:45.592118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a25cf9-3017-46bf-b460-95c3ffd3fb38 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:45.592118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a25cf9-3017-46bf-b460-95c3ffd3fb38 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:45.593448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 641/1302] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:57:45.598656 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e2f91-dece-457e-b0b2-94daeeb971e4 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:45.599026 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e2f91-dece-457e-b0b2-94daeeb971e4 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:45.599201 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e2f91-dece-457e-b0b2-94daeeb971e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:45.617455 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-c09e2f91-dece-457e-b0b2-94daeeb971e4 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:57:45.618663 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-c09e2f91-dece-457e-b0b2-94daeeb971e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:57:45.682891 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 662/1303] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:57:45 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:57:45.947927 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a25cf9-3017-46bf-b460-95c3ffd3fb38 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:46.040210 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:46.047808 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a25cf9-3017-46bf-b460-95c3ffd3fb38 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:46.079602 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e325f-a992-488f-bd84-9fa44184714a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:46.080221 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e325f-a992-488f-bd84-9fa44184714a None admin] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:46.080497 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e325f-a992-488f-bd84-9fa44184714a None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:46.080766 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e325f-a992-488f-bd84-9fa44184714a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:46.081336 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e325f-a992-488f-bd84-9fa44184714a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:46.082712 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e325f-a992-488f-bd84-9fa44184714a None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:46.091693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application [None req-b45e325f-a992-488f-bd84-9fa44184714a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 01:57:46.091693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:46.091693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:57:46.091693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:57:46.091693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:57:46.091693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:57:46.091693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:57:46.091693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:57:46.091693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 01:57:46.091693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:46.091693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:57:46.091693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:46.091693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:57:46.091693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:57:46.091693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:57:46.091693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:57:46.091693 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:57:46.092865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:57:46.092865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:57:46.092865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:57:46.092865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:57:46.092865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:57:46.092865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:57:46.092865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:57:46.092865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:57:46.092865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:57:46.092865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:57:46.092865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:57:46.092865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:57:46.092865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:57:46.092865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:57:46.092865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:57:46.092865 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:57:46.093761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:57:46.093761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application with Lock( Sep 14 01:57:46.093761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:57:46.093761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:57:46.093761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:57:46.093761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:57:46.093761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:57:46.093761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:57:46.093761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:57:46.093761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:57:46.093761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:57:46.093761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:57:46.093761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 01:57:46.093761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: ERROR keystone.server.flask.application Sep 14 01:57:46.093761 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 642/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Sep 14 01:57:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:57:46.098738 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e2f91-dece-457e-b0b2-94daeeb971e4 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:46.171630 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:46.193203 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e2f91-dece-457e-b0b2-94daeeb971e4 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:46.225931 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990da222-248c-4f40-a03e-c0b3bce9676b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:46.226554 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990da222-248c-4f40-a03e-c0b3bce9676b None admin] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:46.226832 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990da222-248c-4f40-a03e-c0b3bce9676b None admin] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:46.227098 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990da222-248c-4f40-a03e-c0b3bce9676b None admin] PATH_INFO: `/v3/services` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:46.227886 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990da222-248c-4f40-a03e-c0b3bce9676b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:57:46.228200 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990da222-248c-4f40-a03e-c0b3bce9676b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:46.229612 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990da222-248c-4f40-a03e-c0b3bce9676b None admin] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:46.234571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 663/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Sep 14 01:57:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:57:46.240169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e325f-a992-488f-bd84-9fa44184714a None admin] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:57:46.335741 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e325f-a992-488f-bd84-9fa44184714a None admin] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:57:46.368785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc5231a-b6bc-4f3b-aa73-fa2bbed5edf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25374d78618f4e32be8780901a31679f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:57:46.369385 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5231a-b6bc-4f3b-aa73-fa2bbed5edf0 None admin] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:57:46.369657 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5231a-b6bc-4f3b-aa73-fa2bbed5edf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:57:46.369919 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5231a-b6bc-4f3b-aa73-fa2bbed5edf0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:57:46.370587 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc5231a-b6bc-4f3b-aa73-fa2bbed5edf0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:57:46.371925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc5231a-b6bc-4f3b-aa73-fa2bbed5edf0 None admin] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:57:46.391289 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 643/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 14 01:57:46 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:57:56.232164 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=30 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:57:56.387448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:58:50.304955 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f624010-f0a7-4b64-aa21-794c50ec790f None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:58:50.305461 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f624010-f0a7-4b64-aa21-794c50ec790f None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:58:50.305554 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f624010-f0a7-4b64-aa21-794c50ec790f None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:58:50.306853 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 664/1307] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:58:50.318986 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73ccac-5dd0-4241-867f-f21e2c76fd39 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:58:50.320999 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73ccac-5dd0-4241-867f-f21e2c76fd39 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:58:50.322298 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73ccac-5dd0-4241-867f-f21e2c76fd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:58:50.453590 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-1b73ccac-5dd0-4241-867f-f21e2c76fd39 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:58:50.455450 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-1b73ccac-5dd0-4241-867f-f21e2c76fd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:58:50.579378 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 644/1308] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:58:50.896277 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1be956-a8f2-403f-83f1-eb265819d4d9 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:58:50.896557 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1be956-a8f2-403f-83f1-eb265819d4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:58:50.896830 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1be956-a8f2-403f-83f1-eb265819d4d9 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:58:50.898586 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 665/1309] 10.222.0.15 () {54 vars in 965 bytes} [Wed Sep 14 01:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:58:50.904975 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdb6e4-deef-496e-8916-4bad05918f3b None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:58:50.905613 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdb6e4-deef-496e-8916-4bad05918f3b None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:58:50.906165 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdb6e4-deef-496e-8916-4bad05918f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:58:51.003222 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-6bfdb6e4-deef-496e-8916-4bad05918f3b None None] MFA Rules not processed for user `bf86028f85af444380293d2387928eda`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:58:51.004666 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-6bfdb6e4-deef-496e-8916-4bad05918f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:58:51.028786 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:58:51.112496 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1be956-a8f2-403f-83f1-eb265819d4d9 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:58:51.114254 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-9d1be956-a8f2-403f-83f1-eb265819d4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:58:51.118578 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 645/1310] 10.222.0.15 () {58 vars in 1062 bytes} [Wed Sep 14 01:58:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:58:51.227857 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1be956-a8f2-403f-83f1-eb265819d4d9 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:58:51.309273 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11cbe8-be6b-4ae5-a7ff-5cb0441859d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf86028f85af444380293d2387928eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:58:51.309926 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11cbe8-be6b-4ae5-a7ff-5cb0441859d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:58:51.310247 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11cbe8-be6b-4ae5-a7ff-5cb0441859d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:58:51.310353 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11cbe8-be6b-4ae5-a7ff-5cb0441859d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:58:51.311969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-7d11cbe8-be6b-4ae5-a7ff-5cb0441859d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:58:51.370429 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:58:51.430952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11cbe8-be6b-4ae5-a7ff-5cb0441859d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:58:51.432581 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11cbe8-be6b-4ae5-a7ff-5cb0441859d8 service neutron] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:58:51.454805 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 666/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 01:58:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:58:54.075520 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507c2a2-47be-4740-8dc2-d0c1419b33f6 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:58:54.076042 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507c2a2-47be-4740-8dc2-d0c1419b33f6 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:58:54.076042 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507c2a2-47be-4740-8dc2-d0c1419b33f6 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:58:54.077341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 646/1312] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:58:54.082790 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d07730-87b2-462b-a868-97c130051a53 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:58:54.083205 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d07730-87b2-462b-a868-97c130051a53 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:58:54.083322 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d07730-87b2-462b-a868-97c130051a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:58:54.152965 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-c6d07730-87b2-462b-a868-97c130051a53 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:58:54.154429 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-c6d07730-87b2-462b-a868-97c130051a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:58:54.249422 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 667/1313] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:58:54.532580 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507c2a2-47be-4740-8dc2-d0c1419b33f6 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:58:54.534223 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-b507c2a2-47be-4740-8dc2-d0c1419b33f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:58:54.637296 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507c2a2-47be-4740-8dc2-d0c1419b33f6 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:58:54.706929 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:58:54.709293 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a429d8c-491b-4c1f-8a64-f4d3498d1ed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf86028f85af444380293d2387928eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:58:54.710142 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a429d8c-491b-4c1f-8a64-f4d3498d1ed2 service neutron] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:58:54.710603 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a429d8c-491b-4c1f-8a64-f4d3498d1ed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:58:54.711060 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a429d8c-491b-4c1f-8a64-f4d3498d1ed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:58:54.712656 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-1a429d8c-491b-4c1f-8a64-f4d3498d1ed2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:58:54.827763 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a429d8c-491b-4c1f-8a64-f4d3498d1ed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:58:54.829201 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a429d8c-491b-4c1f-8a64-f4d3498d1ed2 service neutron] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:58:54.847351 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 647/1314] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 01:58:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:58:57.267888 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f39c0-cbe9-458b-a8dd-eb55cb735c58 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:58:57.268455 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f39c0-cbe9-458b-a8dd-eb55cb735c58 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:58:57.268455 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f39c0-cbe9-458b-a8dd-eb55cb735c58 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:58:57.269756 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 668/1315] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:58:57.275683 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff29938-1878-4afd-a370-dc42caef0c8f None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:58:57.276088 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff29938-1878-4afd-a370-dc42caef0c8f None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:58:57.276224 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff29938-1878-4afd-a370-dc42caef0c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:58:57.340789 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-0ff29938-1878-4afd-a370-dc42caef0c8f None None] MFA Rules not processed for user `77fa2fe74be84241a0d5150ae86101fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:58:57.341965 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-0ff29938-1878-4afd-a370-dc42caef0c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:58:57.392941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:58:57.437215 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 648/1316] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:58:57 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:58:57.715364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f39c0-cbe9-458b-a8dd-eb55cb735c58 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:58:57.716921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-589f39c0-cbe9-458b-a8dd-eb55cb735c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:58:57.727270 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:58:57.818698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f39c0-cbe9-458b-a8dd-eb55cb735c58 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:58:57.896579 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626fe617-8f5d-4eb8-a74b-9baf625cbfe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf86028f85af444380293d2387928eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:58:57.897173 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fe617-8f5d-4eb8-a74b-9baf625cbfe3 service neutron] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:58:57.897446 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fe617-8f5d-4eb8-a74b-9baf625cbfe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:58:57.897709 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fe617-8f5d-4eb8-a74b-9baf625cbfe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:58:57.899851 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-626fe617-8f5d-4eb8-a74b-9baf625cbfe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:58:57.960127 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:58:58.005457 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626fe617-8f5d-4eb8-a74b-9baf625cbfe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:58:58.006931 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626fe617-8f5d-4eb8-a74b-9baf625cbfe3 service neutron] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:58:58.024709 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 669/1317] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 01:58:57 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:59:00.684423 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd05b2-7459-4c8c-af05-b9e7b2d415e2 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:00.684933 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd05b2-7459-4c8c-af05-b9e7b2d415e2 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:00.684933 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd05b2-7459-4c8c-af05-b9e7b2d415e2 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:00.686260 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 649/1318] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:00.691362 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e9c40-85da-4125-9fa7-506fb352fbb5 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:00.691763 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e9c40-85da-4125-9fa7-506fb352fbb5 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:00.691902 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e9c40-85da-4125-9fa7-506fb352fbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:00.756421 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-424e9c40-85da-4125-9fa7-506fb352fbb5 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:00.758141 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-424e9c40-85da-4125-9fa7-506fb352fbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:00.851298 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 670/1319] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:59:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:59:01.167492 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620db2da-1f87-4636-8cf9-bbc1c348e0a2 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:01.167800 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620db2da-1f87-4636-8cf9-bbc1c348e0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:01.168070 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620db2da-1f87-4636-8cf9-bbc1c348e0a2 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:01.169415 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 650/1320] 10.222.0.15 () {54 vars in 965 bytes} [Wed Sep 14 01:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:01.175293 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84baeb4-63a3-4153-a3b4-b83b43008797 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:01.175580 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84baeb4-63a3-4153-a3b4-b83b43008797 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:01.175850 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84baeb4-63a3-4153-a3b4-b83b43008797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:01.197517 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:01.250578 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-e84baeb4-63a3-4153-a3b4-b83b43008797 None None] MFA Rules not processed for user `bf86028f85af444380293d2387928eda`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:01.251768 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-e84baeb4-63a3-4153-a3b4-b83b43008797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:01.325755 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 671/1321] 10.222.0.15 () {58 vars in 1062 bytes} [Wed Sep 14 01:59:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:59:01.334599 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620db2da-1f87-4636-8cf9-bbc1c348e0a2 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:59:01.336163 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-620db2da-1f87-4636-8cf9-bbc1c348e0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:01.436925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620db2da-1f87-4636-8cf9-bbc1c348e0a2 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:59:01.517236 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ccb97-2f87-4375-b6de-5ecc9066d3ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf86028f85af444380293d2387928eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:59:01.517837 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ccb97-2f87-4375-b6de-5ecc9066d3ca service neutron] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:01.518138 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ccb97-2f87-4375-b6de-5ecc9066d3ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:01.518406 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ccb97-2f87-4375-b6de-5ecc9066d3ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:01.519997 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-b84ccb97-2f87-4375-b6de-5ecc9066d3ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:01.527588 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:01.633333 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ccb97-2f87-4375-b6de-5ecc9066d3ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:59:01.634795 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ccb97-2f87-4375-b6de-5ecc9066d3ca service neutron] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:59:01.653149 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 651/1322] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 01:59:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:59:04.903388 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6e551-42e3-40fa-a423-a2bbdb2fc22c None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:04.903952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6e551-42e3-40fa-a423-a2bbdb2fc22c None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:04.903952 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6e551-42e3-40fa-a423-a2bbdb2fc22c None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:04.905313 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 672/1323] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:04.911290 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ec080-a0fe-4dcd-ba94-418123d2e203 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:04.911866 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ec080-a0fe-4dcd-ba94-418123d2e203 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:04.911866 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ec080-a0fe-4dcd-ba94-418123d2e203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:04.986670 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-283ec080-a0fe-4dcd-ba94-418123d2e203 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:04.987896 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-283ec080-a0fe-4dcd-ba94-418123d2e203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:05.050740 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:05.093092 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 652/1324] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:59:05.369039 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6e551-42e3-40fa-a423-a2bbdb2fc22c None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:59:05.370609 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-69a6e551-42e3-40fa-a423-a2bbdb2fc22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:05.491235 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6e551-42e3-40fa-a423-a2bbdb2fc22c None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:59:05.504542 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:05.565528 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becf1eb-6ad2-4304-8ced-3cee2c23f5fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf86028f85af444380293d2387928eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:59:05.566167 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becf1eb-6ad2-4304-8ced-3cee2c23f5fe service neutron] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:05.566449 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becf1eb-6ad2-4304-8ced-3cee2c23f5fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:05.566719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becf1eb-6ad2-4304-8ced-3cee2c23f5fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:05.568309 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-2becf1eb-6ad2-4304-8ced-3cee2c23f5fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:05.678676 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2becf1eb-6ad2-4304-8ced-3cee2c23f5fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:59:05.680229 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2becf1eb-6ad2-4304-8ced-3cee2c23f5fe service neutron] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:59:05.697725 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 673/1325] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 01:59:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:59:07.872410 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095e087-74fc-4bdc-af9a-0717b79b100e None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:07.873015 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095e087-74fc-4bdc-af9a-0717b79b100e None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:07.873015 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095e087-74fc-4bdc-af9a-0717b79b100e None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:07.874466 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 653/1326] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:07.879799 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab661a2-5546-4edf-87e7-1b662af48a40 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:07.880213 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab661a2-5546-4edf-87e7-1b662af48a40 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:07.880483 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab661a2-5546-4edf-87e7-1b662af48a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:07.927967 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:07.955775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-cab661a2-5546-4edf-87e7-1b662af48a40 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:07.956979 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-cab661a2-5546-4edf-87e7-1b662af48a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:08.056015 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 674/1327] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:59:08.335169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095e087-74fc-4bdc-af9a-0717b79b100e None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:59:08.337359 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-9095e087-74fc-4bdc-af9a-0717b79b100e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:08.446909 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095e087-74fc-4bdc-af9a-0717b79b100e None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:59:08.520081 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879519d2-1f44-45f1-bec9-7fe34cc5026c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf86028f85af444380293d2387928eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:59:08.520681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879519d2-1f44-45f1-bec9-7fe34cc5026c service neutron] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:08.520955 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879519d2-1f44-45f1-bec9-7fe34cc5026c service neutron] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:08.521220 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879519d2-1f44-45f1-bec9-7fe34cc5026c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:08.523325 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-879519d2-1f44-45f1-bec9-7fe34cc5026c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:08.541022 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:08.635664 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879519d2-1f44-45f1-bec9-7fe34cc5026c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:59:08.637096 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879519d2-1f44-45f1-bec9-7fe34cc5026c service neutron] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:59:08.655118 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 654/1328] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 01:59:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:59:10.870149 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be3c18-f6a9-4f80-8b56-6a347357db83 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:10.870651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be3c18-f6a9-4f80-8b56-6a347357db83 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:10.870651 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be3c18-f6a9-4f80-8b56-6a347357db83 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:10.871948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 675/1329] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:10.877282 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9725e-bafc-4212-a102-7c00e1fe6470 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:10.877711 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9725e-bafc-4212-a102-7c00e1fe6470 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:10.877837 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9725e-bafc-4212-a102-7c00e1fe6470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:10.944125 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-70b9725e-bafc-4212-a102-7c00e1fe6470 None None] MFA Rules not processed for user `77fa2fe74be84241a0d5150ae86101fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:10.945363 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-70b9725e-bafc-4212-a102-7c00e1fe6470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:11.009040 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:11.043171 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 655/1330] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:59:10 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:59:11.324468 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be3c18-f6a9-4f80-8b56-6a347357db83 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:59:11.326038 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-f8be3c18-f6a9-4f80-8b56-6a347357db83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:11.432155 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be3c18-f6a9-4f80-8b56-6a347357db83 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:59:11.453198 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:11.506458 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325992e4-b90f-4ed8-aaf6-024d2484b8b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf86028f85af444380293d2387928eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:59:11.507057 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325992e4-b90f-4ed8-aaf6-024d2484b8b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:11.507866 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325992e4-b90f-4ed8-aaf6-024d2484b8b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:11.508133 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325992e4-b90f-4ed8-aaf6-024d2484b8b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:11.509758 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-325992e4-b90f-4ed8-aaf6-024d2484b8b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:11.619605 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325992e4-b90f-4ed8-aaf6-024d2484b8b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:59:11.621046 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325992e4-b90f-4ed8-aaf6-024d2484b8b0 service neutron] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:59:11.643178 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 676/1331] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 01:59:11 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:59:14.041854 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feaefea-64b8-4b82-872f-885be80a41c8 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:14.042450 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feaefea-64b8-4b82-872f-885be80a41c8 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:14.042450 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feaefea-64b8-4b82-872f-885be80a41c8 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:14.043799 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 656/1332] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:14.048911 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037875a-4c48-44df-a7ae-d972fb908f34 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:14.049305 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037875a-4c48-44df-a7ae-d972fb908f34 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:14.049467 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037875a-4c48-44df-a7ae-d972fb908f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:14.112633 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:14.115579 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-3037875a-4c48-44df-a7ae-d972fb908f34 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:14.117919 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-3037875a-4c48-44df-a7ae-d972fb908f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:14.210904 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 677/1333] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:59:14.497387 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feaefea-64b8-4b82-872f-885be80a41c8 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:59:14.498946 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-5feaefea-64b8-4b82-872f-885be80a41c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:14.600307 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feaefea-64b8-4b82-872f-885be80a41c8 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:59:14.673026 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22801d-1fa9-41a0-b19a-f1a281a9342e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf86028f85af444380293d2387928eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:59:14.673651 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22801d-1fa9-41a0-b19a-f1a281a9342e service neutron] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:14.673925 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22801d-1fa9-41a0-b19a-f1a281a9342e service neutron] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:14.674220 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22801d-1fa9-41a0-b19a-f1a281a9342e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:14.675803 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-3a22801d-1fa9-41a0-b19a-f1a281a9342e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:14.703618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:14.789364 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22801d-1fa9-41a0-b19a-f1a281a9342e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:59:14.790840 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22801d-1fa9-41a0-b19a-f1a281a9342e service neutron] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:59:14.810830 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 657/1334] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 01:59:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:59:16.996775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8930b3-68fd-4682-a7d0-cf68e76f9f13 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:16.997307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8930b3-68fd-4682-a7d0-cf68e76f9f13 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:16.997307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8930b3-68fd-4682-a7d0-cf68e76f9f13 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:16.998680 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 678/1335] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:17.004009 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f24f8-0a66-4877-a683-296f3d312f2b None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:17.004681 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f24f8-0a66-4877-a683-296f3d312f2b None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:17.004835 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f24f8-0a66-4877-a683-296f3d312f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:17.071846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-7e8f24f8-0a66-4877-a683-296f3d312f2b None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:17.071846 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-7e8f24f8-0a66-4877-a683-296f3d312f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:17.145000 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:17.167275 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 658/1336] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:59:17.466307 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8930b3-68fd-4682-a7d0-cf68e76f9f13 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:59:17.467860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-bc8930b3-68fd-4682-a7d0-cf68e76f9f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:17.576698 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8930b3-68fd-4682-a7d0-cf68e76f9f13 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:59:17.604699 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:17.654486 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd4f9a-5ee5-4f6c-8706-d0ef0adb69cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf86028f85af444380293d2387928eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:59:17.655010 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd4f9a-5ee5-4f6c-8706-d0ef0adb69cf service neutron] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:17.655281 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd4f9a-5ee5-4f6c-8706-d0ef0adb69cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:17.655545 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd4f9a-5ee5-4f6c-8706-d0ef0adb69cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:17.657164 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-28fd4f9a-5ee5-4f6c-8706-d0ef0adb69cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:17.763757 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd4f9a-5ee5-4f6c-8706-d0ef0adb69cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:59:17.765190 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd4f9a-5ee5-4f6c-8706-d0ef0adb69cf service neutron] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:59:17.783145 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 679/1337] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 01:59:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:59:20.250188 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685814e3-a742-4a65-8db8-4459131ee8cb None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:20.250727 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685814e3-a742-4a65-8db8-4459131ee8cb None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:20.250727 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685814e3-a742-4a65-8db8-4459131ee8cb None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:20.252077 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 659/1338] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:20.257515 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07889b58-5d69-4185-a709-f71e0c13e257 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:20.257929 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07889b58-5d69-4185-a709-f71e0c13e257 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:20.258151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07889b58-5d69-4185-a709-f71e0c13e257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:20.322714 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-07889b58-5d69-4185-a709-f71e0c13e257 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:20.323908 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-07889b58-5d69-4185-a709-f71e0c13e257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:20.333192 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:20.422747 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 680/1339] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:59:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:59:20.699946 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685814e3-a742-4a65-8db8-4459131ee8cb None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:59:20.701477 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-685814e3-a742-4a65-8db8-4459131ee8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:20.802038 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685814e3-a742-4a65-8db8-4459131ee8cb None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:59:20.876992 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee047a-477c-44a2-a961-97e8280b6617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf86028f85af444380293d2387928eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:59:20.877589 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee047a-477c-44a2-a961-97e8280b6617 service neutron] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:20.877870 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee047a-477c-44a2-a961-97e8280b6617 service neutron] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:20.878162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee047a-477c-44a2-a961-97e8280b6617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:20.880017 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-e0ee047a-477c-44a2-a961-97e8280b6617 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:20.920328 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:20.992472 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee047a-477c-44a2-a961-97e8280b6617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:59:20.993837 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee047a-477c-44a2-a961-97e8280b6617 service neutron] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:59:21.012117 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 660/1340] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 01:59:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:59:25.025616 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2d391-5ccd-4660-a86c-1b8584c518e8 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:25.026151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2d391-5ccd-4660-a86c-1b8584c518e8 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:25.026250 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2d391-5ccd-4660-a86c-1b8584c518e8 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:25.027669 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 681/1341] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:25.032941 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d294b-cf23-477d-8fe9-4c4c4e15f222 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:25.033389 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d294b-cf23-477d-8fe9-4c4c4e15f222 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:25.033563 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d294b-cf23-477d-8fe9-4c4c4e15f222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:25.103910 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-404d294b-cf23-477d-8fe9-4c4c4e15f222 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:25.105125 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-404d294b-cf23-477d-8fe9-4c4c4e15f222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:25.192974 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:25.206554 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 661/1342] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:59:25.487603 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2d391-5ccd-4660-a86c-1b8584c518e8 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:59:25.489200 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-c1b2d391-5ccd-4660-a86c-1b8584c518e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:25.590571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2d391-5ccd-4660-a86c-1b8584c518e8 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:59:25.628775 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:25.665388 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e30e3-2391-4243-915c-f198026bb2a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf86028f85af444380293d2387928eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:59:25.665906 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e30e3-2391-4243-915c-f198026bb2a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:25.666206 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e30e3-2391-4243-915c-f198026bb2a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:25.666475 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e30e3-2391-4243-915c-f198026bb2a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:25.668043 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-a26e30e3-2391-4243-915c-f198026bb2a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:25.778593 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e30e3-2391-4243-915c-f198026bb2a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:59:25.780034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e30e3-2391-4243-915c-f198026bb2a5 service neutron] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:59:25.798628 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 682/1343] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 01:59:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:59:28.517658 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb0aa2-7dab-42b1-9902-d61f5dbddc2b None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:28.518221 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb0aa2-7dab-42b1-9902-d61f5dbddc2b None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:28.518221 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb0aa2-7dab-42b1-9902-d61f5dbddc2b None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:28.519751 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 662/1344] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:28.525239 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94dc09-ba2c-4123-9add-417366cac927 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:28.525666 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94dc09-ba2c-4123-9add-417366cac927 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:28.525761 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94dc09-ba2c-4123-9add-417366cac927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:28.591361 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-4e94dc09-ba2c-4123-9add-417366cac927 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:28.592558 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-4e94dc09-ba2c-4123-9add-417366cac927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:28.610284 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:28.695850 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 683/1345] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:59:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:59:28.991637 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb0aa2-7dab-42b1-9902-d61f5dbddc2b None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:59:28.993279 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-64cb0aa2-7dab-42b1-9902-d61f5dbddc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:29.122408 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb0aa2-7dab-42b1-9902-d61f5dbddc2b None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:59:29.199386 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6d66f-64e6-4744-b34f-2f315a9b3df0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf86028f85af444380293d2387928eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:59:29.199994 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6d66f-64e6-4744-b34f-2f315a9b3df0 service neutron] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:29.200278 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6d66f-64e6-4744-b34f-2f315a9b3df0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:29.200547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6d66f-64e6-4744-b34f-2f315a9b3df0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:29.202197 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-0ba6d66f-64e6-4744-b34f-2f315a9b3df0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:29.247124 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:29.316052 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6d66f-64e6-4744-b34f-2f315a9b3df0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:59:29.317529 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6d66f-64e6-4744-b34f-2f315a9b3df0 service neutron] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:59:29.337785 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 663/1346] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 01:59:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:59:33.537507 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9c34f-1d5e-41be-ae38-64d5f0dbe744 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:33.538655 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9c34f-1d5e-41be-ae38-64d5f0dbe744 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:33.538655 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9c34f-1d5e-41be-ae38-64d5f0dbe744 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:33.539924 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 684/1347] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:59:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:33.545160 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7f5ff-6734-4d02-938c-666f618651f5 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:33.545564 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7f5ff-6734-4d02-938c-666f618651f5 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:33.545717 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7f5ff-6734-4d02-938c-666f618651f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:33.613803 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-48b7f5ff-6734-4d02-938c-666f618651f5 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:33.615403 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-48b7f5ff-6734-4d02-938c-666f618651f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:33.707522 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:33.709784 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 664/1348] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:59:33.988312 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f9c34f-1d5e-41be-ae38-64d5f0dbe744 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:59:33.990027 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-a5f9c34f-1d5e-41be-ae38-64d5f0dbe744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:34.097200 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f9c34f-1d5e-41be-ae38-64d5f0dbe744 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:59:34.146557 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:34.171989 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64de165-8ec2-45c5-a8d8-ab218147c412 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf86028f85af444380293d2387928eda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:59:34.172586 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64de165-8ec2-45c5-a8d8-ab218147c412 service neutron] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:34.172887 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64de165-8ec2-45c5-a8d8-ab218147c412 service neutron] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:34.173153 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64de165-8ec2-45c5-a8d8-ab218147c412 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:34.174766 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-e64de165-8ec2-45c5-a8d8-ab218147c412 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:34.282232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64de165-8ec2-45c5-a8d8-ab218147c412 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:59:34.283684 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64de165-8ec2-45c5-a8d8-ab218147c412 service neutron] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:59:34.301424 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 685/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 01:59:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:59:39.184739 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc31de-d2e4-471d-8307-31677c77d04c None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:39.185162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc31de-d2e4-471d-8307-31677c77d04c None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:39.185345 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc31de-d2e4-471d-8307-31677c77d04c None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:39.188042 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 665/1350] 10.222.0.15 () {54 vars in 921 bytes} [Wed Sep 14 01:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:39.191987 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccc746-f525-4b7f-9a27-526ce14432ca None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:39.192442 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccc746-f525-4b7f-9a27-526ce14432ca None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:39.192735 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccc746-f525-4b7f-9a27-526ce14432ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:39.298772 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-bfccc746-f525-4b7f-9a27-526ce14432ca None None] MFA Rules not processed for user `4da8e712cd9b454fa035f93785775fd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:39.299241 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-bfccc746-f525-4b7f-9a27-526ce14432ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:39.327181 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:39.444441 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 686/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Sep 14 01:59:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:59:43.836890 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2669d-7cbc-4f1e-94c7-af238dee597b None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:43.837736 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2669d-7cbc-4f1e-94c7-af238dee597b None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:43.838032 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2669d-7cbc-4f1e-94c7-af238dee597b None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:43.839582 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 666/1352] 10.222.0.15 () {54 vars in 921 bytes} [Wed Sep 14 01:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:43.844472 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614e0b7-6e38-4bfc-a49c-2770a96f2f3e None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:43.844921 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614e0b7-6e38-4bfc-a49c-2770a96f2f3e None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:43.845225 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614e0b7-6e38-4bfc-a49c-2770a96f2f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:43.929611 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-9614e0b7-6e38-4bfc-a49c-2770a96f2f3e None None] MFA Rules not processed for user `4da8e712cd9b454fa035f93785775fd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:43.930905 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-9614e0b7-6e38-4bfc-a49c-2770a96f2f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:44.008430 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:44.056522 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 687/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Sep 14 01:59:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:59:45.354549 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093816c5-574c-4e67-b923-69382c6fa1d7 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:45.355105 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093816c5-574c-4e67-b923-69382c6fa1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:45.355105 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093816c5-574c-4e67-b923-69382c6fa1d7 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:45.356471 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 667/1354] 10.222.0.15 () {54 vars in 921 bytes} [Wed Sep 14 01:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:45.361487 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd041fa-5ca3-4e40-b3c3-7a1d0424d9fa None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:45.361914 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd041fa-5ca3-4e40-b3c3-7a1d0424d9fa None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:45.362054 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd041fa-5ca3-4e40-b3c3-7a1d0424d9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:45.442753 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-4cd041fa-5ca3-4e40-b3c3-7a1d0424d9fa None None] MFA Rules not processed for user `4da8e712cd9b454fa035f93785775fd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:45.444515 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-4cd041fa-5ca3-4e40-b3c3-7a1d0424d9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:45.558263 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 688/1355] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Sep 14 01:59:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:59:48.972869 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be4b33-43b0-41d6-bb7e-62683ac225d7 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:48.973381 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be4b33-43b0-41d6-bb7e-62683ac225d7 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:48.973381 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be4b33-43b0-41d6-bb7e-62683ac225d7 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:48.974752 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 668/1356] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:48.980946 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770f5b3-5a89-497d-9a8e-140deb16ad29 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:48.981413 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770f5b3-5a89-497d-9a8e-140deb16ad29 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:48.981497 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770f5b3-5a89-497d-9a8e-140deb16ad29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:49.026799 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:49.067283 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-0770f5b3-5a89-497d-9a8e-140deb16ad29 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:49.068477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-0770f5b3-5a89-497d-9a8e-140deb16ad29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:49.172732 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 689/1357] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:59:49.468030 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5bad0-c83e-40b8-97de-beee1053200d None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:49.468416 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5bad0-c83e-40b8-97de-beee1053200d None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:49.468794 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5bad0-c83e-40b8-97de-beee1053200d None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:49.470258 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 669/1358] 10.222.0.15 () {54 vars in 962 bytes} [Wed Sep 14 01:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:49.485746 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3960e18-e7ea-4b7b-a44c-9311c7ba789f None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:49.486127 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3960e18-e7ea-4b7b-a44c-9311c7ba789f None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:49.486508 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3960e18-e7ea-4b7b-a44c-9311c7ba789f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:49.568151 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-c3960e18-e7ea-4b7b-a44c-9311c7ba789f None None] MFA Rules not processed for user `f721a3a4eca941989685448a87bb9a19`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:49.569861 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-c3960e18-e7ea-4b7b-a44c-9311c7ba789f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:49.641977 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:49.719049 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 690/1359] 10.222.0.15 () {58 vars in 1059 bytes} [Wed Sep 14 01:59:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:59:49.727774 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5bad0-c83e-40b8-97de-beee1053200d None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:59:49.729341 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-28d5bad0-c83e-40b8-97de-beee1053200d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:49.891063 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5bad0-c83e-40b8-97de-beee1053200d None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:59:49.968002 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:50.009463 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555a7b7-654e-4c93-ac74-6897c2ca8cc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f721a3a4eca941989685448a87bb9a19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:59:50.010157 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555a7b7-654e-4c93-ac74-6897c2ca8cc1 service nova] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:50.010407 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555a7b7-654e-4c93-ac74-6897c2ca8cc1 service nova] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:50.010633 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555a7b7-654e-4c93-ac74-6897c2ca8cc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:50.012225 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-7555a7b7-654e-4c93-ac74-6897c2ca8cc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:50.115936 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555a7b7-654e-4c93-ac74-6897c2ca8cc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:59:50.117534 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555a7b7-654e-4c93-ac74-6897c2ca8cc1 service nova] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:59:50.137435 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 670/1360] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 01:59:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:59:52.532018 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c453d2-5a70-4c92-b86c-4f49a0b4cdd0 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:52.532018 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c453d2-5a70-4c92-b86c-4f49a0b4cdd0 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:52.532823 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c453d2-5a70-4c92-b86c-4f49a0b4cdd0 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:52.539373 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebf670-e824-4f8c-9e52-e85d28c9adba None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:52.539972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebf670-e824-4f8c-9e52-e85d28c9adba None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:52.539972 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebf670-e824-4f8c-9e52-e85d28c9adba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:52.542486 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 691/1361] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:59:52 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:52.609771 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-ffebf670-e824-4f8c-9e52-e85d28c9adba None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:52.611000 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-ffebf670-e824-4f8c-9e52-e85d28c9adba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:52.649936 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:52.722349 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 671/1362] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:59:53.012936 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c453d2-5a70-4c92-b86c-4f49a0b4cdd0 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:59:53.014502 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-18c453d2-5a70-4c92-b86c-4f49a0b4cdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:53.203201 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c453d2-5a70-4c92-b86c-4f49a0b4cdd0 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:59:53.214505 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:53.336667 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a7b90-56c8-4368-b0ab-7da48b638e67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f721a3a4eca941989685448a87bb9a19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:59:53.337570 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a7b90-56c8-4368-b0ab-7da48b638e67 service nova] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:53.337845 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a7b90-56c8-4368-b0ab-7da48b638e67 service nova] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:53.338179 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a7b90-56c8-4368-b0ab-7da48b638e67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:53.339776 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-ce8a7b90-56c8-4368-b0ab-7da48b638e67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:53.436339 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a7b90-56c8-4368-b0ab-7da48b638e67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:59:53.437962 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a7b90-56c8-4368-b0ab-7da48b638e67 service nova] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:59:53.456142 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 692/1363] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 01:59:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:59:55.938541 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cffa45-af11-4986-9c7f-6df885987aa6 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:55.939062 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cffa45-af11-4986-9c7f-6df885987aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:55.939062 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cffa45-af11-4986-9c7f-6df885987aa6 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:55.940611 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 672/1364] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:55.945639 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d900b91-534c-419f-b517-77b5c0182d60 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:55.946065 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d900b91-534c-419f-b517-77b5c0182d60 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:55.946276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d900b91-534c-419f-b517-77b5c0182d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:55.962583 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:56.017424 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-5d900b91-534c-419f-b517-77b5c0182d60 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:56.019392 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-5d900b91-534c-419f-b517-77b5c0182d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:56.124233 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 693/1365] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:59:56.410065 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394aeec-76a8-4232-b08e-a21b023041ba None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:56.410357 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394aeec-76a8-4232-b08e-a21b023041ba None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:56.410619 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394aeec-76a8-4232-b08e-a21b023041ba None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:56.411977 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 673/1366] 10.222.0.15 () {54 vars in 962 bytes} [Wed Sep 14 01:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:56.418081 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c3bd9-1c52-4c6c-967d-40bef7a8fab3 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:56.418534 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c3bd9-1c52-4c6c-967d-40bef7a8fab3 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:56.418804 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c3bd9-1c52-4c6c-967d-40bef7a8fab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:56.496945 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-0c4c3bd9-1c52-4c6c-967d-40bef7a8fab3 None None] MFA Rules not processed for user `f721a3a4eca941989685448a87bb9a19`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:56.498254 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-0c4c3bd9-1c52-4c6c-967d-40bef7a8fab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:56.535189 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:56.621206 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 694/1367] 10.222.0.15 () {58 vars in 1059 bytes} [Wed Sep 14 01:59:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:59:56.629010 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394aeec-76a8-4232-b08e-a21b023041ba None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:59:56.630803 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-6394aeec-76a8-4232-b08e-a21b023041ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:56.788320 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394aeec-76a8-4232-b08e-a21b023041ba None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:59:56.809353 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:56.908750 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920dc50-d2f2-4d23-8ed5-99bafa962691 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f721a3a4eca941989685448a87bb9a19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:59:56.909345 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920dc50-d2f2-4d23-8ed5-99bafa962691 service nova] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:56.909618 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920dc50-d2f2-4d23-8ed5-99bafa962691 service nova] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:56.909877 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920dc50-d2f2-4d23-8ed5-99bafa962691 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:56.911487 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-b920dc50-d2f2-4d23-8ed5-99bafa962691 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:57.005594 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920dc50-d2f2-4d23-8ed5-99bafa962691 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:59:57.007755 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920dc50-d2f2-4d23-8ed5-99bafa962691 service nova] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:59:57.030169 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 674/1368] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 01:59:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:59:57.175254 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5f8d1-3db1-4cd6-8c61-280859d4474c None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:57.175860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5f8d1-3db1-4cd6-8c61-280859d4474c None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:57.175860 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5f8d1-3db1-4cd6-8c61-280859d4474c None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:57.177373 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 695/1369] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:57.183508 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c41a6-28c2-4d60-aa83-810fcb1f451c None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:57.183788 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c41a6-28c2-4d60-aa83-810fcb1f451c None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:57.184070 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c41a6-28c2-4d60-aa83-810fcb1f451c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:57.220950 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:57.265573 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-ff3c41a6-28c2-4d60-aa83-810fcb1f451c None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:57.266979 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-ff3c41a6-28c2-4d60-aa83-810fcb1f451c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:57.386093 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 675/1370] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:59:57.718654 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe35503-7201-49f6-b741-9f01fe39b7cf None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:57.718654 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe35503-7201-49f6-b741-9f01fe39b7cf None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:57.718654 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe35503-7201-49f6-b741-9f01fe39b7cf None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:57.720987 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 696/1371] 10.222.0.15 () {54 vars in 964 bytes} [Wed Sep 14 01:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:57.724545 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63932c56-18f4-4c8e-8325-b381cb071d32 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:57.725352 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63932c56-18f4-4c8e-8325-b381cb071d32 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:57.725626 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63932c56-18f4-4c8e-8325-b381cb071d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:57.810194 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-63932c56-18f4-4c8e-8325-b381cb071d32 None None] MFA Rules not processed for user `c6d70ad7054e40c48f8751a3d4fed787`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:57.811419 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-63932c56-18f4-4c8e-8325-b381cb071d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:57.864177 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:57.898232 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 676/1372] 10.222.0.15 () {58 vars in 1061 bytes} [Wed Sep 14 01:59:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:59:57.906120 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe35503-7201-49f6-b741-9f01fe39b7cf None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:59:57.907650 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-afe35503-7201-49f6-b741-9f01fe39b7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:58.025951 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe35503-7201-49f6-b741-9f01fe39b7cf None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:59:58.076795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:58.117075 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180da46d-0fd7-4b88-bfab-ad36ecbc5d84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6d70ad7054e40c48f8751a3d4fed787', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:59:58.117665 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180da46d-0fd7-4b88-bfab-ad36ecbc5d84 service cinder] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:58.117948 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180da46d-0fd7-4b88-bfab-ad36ecbc5d84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:58.118222 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180da46d-0fd7-4b88-bfab-ad36ecbc5d84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:58.120007 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-180da46d-0fd7-4b88-bfab-ad36ecbc5d84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:58.249226 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180da46d-0fd7-4b88-bfab-ad36ecbc5d84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:59:58.250691 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180da46d-0fd7-4b88-bfab-ad36ecbc5d84 service cinder] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:59:58.269686 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 697/1373] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 01:59:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:59:59.283776 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb61c0-928a-430f-a144-c501f438bd35 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:59.284297 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb61c0-928a-430f-a144-c501f438bd35 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:59.284297 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb61c0-928a-430f-a144-c501f438bd35 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:59.285593 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 677/1374] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 01:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:59:59.291462 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfada97-8bd6-4755-acb0-22b8bb42ff47 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:59:59.291910 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfada97-8bd6-4755-acb0-22b8bb42ff47 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:59:59.292141 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfada97-8bd6-4755-acb0-22b8bb42ff47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:59:59.357861 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-9bfada97-8bd6-4755-acb0-22b8bb42ff47 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:59:59.359362 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-9bfada97-8bd6-4755-acb0-22b8bb42ff47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:59.386348 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:59:59.457645 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 698/1375] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 01:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:59:59.769039 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb61c0-928a-430f-a144-c501f438bd35 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:59:59.770624 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-72bb61c0-928a-430f-a144-c501f438bd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:59:59.937577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb61c0-928a-430f-a144-c501f438bd35 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:59:59.997960 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:00.063356 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276d482-2a2c-499b-a52a-57d274a5b1bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f721a3a4eca941989685448a87bb9a19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 02:00:00.064235 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276d482-2a2c-499b-a52a-57d274a5b1bb service nova] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:00.064518 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276d482-2a2c-499b-a52a-57d274a5b1bb service nova] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:00.064792 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276d482-2a2c-499b-a52a-57d274a5b1bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:00.066439 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-5276d482-2a2c-499b-a52a-57d274a5b1bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:00.159892 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5276d482-2a2c-499b-a52a-57d274a5b1bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 02:00:00.160767 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5276d482-2a2c-499b-a52a-57d274a5b1bb service nova] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 02:00:00.180634 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 678/1376] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 01:59:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 02:00:02.407280 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c689db6-e284-4839-a48c-5b031403b4eb None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:02.407807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c689db6-e284-4839-a48c-5b031403b4eb None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:02.407807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c689db6-e284-4839-a48c-5b031403b4eb None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:02.409290 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 699/1377] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 02:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 02:00:02.415085 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73593d-71b6-4c26-83d1-30c9ed0574d2 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:02.415542 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73593d-71b6-4c26-83d1-30c9ed0574d2 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:02.415639 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73593d-71b6-4c26-83d1-30c9ed0574d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:02.482468 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-0f73593d-71b6-4c26-83d1-30c9ed0574d2 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 02:00:02.483641 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-0f73593d-71b6-4c26-83d1-30c9ed0574d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:02.491335 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:02.585577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 679/1378] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 02:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 02:00:02.872606 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c689db6-e284-4839-a48c-5b031403b4eb None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 02:00:02.874166 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-8c689db6-e284-4839-a48c-5b031403b4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:03.030595 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c689db6-e284-4839-a48c-5b031403b4eb None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 02:00:03.034682 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:03.154989 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608721e6-a345-472a-8350-e9c078690299 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f721a3a4eca941989685448a87bb9a19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 02:00:03.155497 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608721e6-a345-472a-8350-e9c078690299 service nova] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:03.155771 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608721e6-a345-472a-8350-e9c078690299 service nova] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:03.156034 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608721e6-a345-472a-8350-e9c078690299 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:03.157596 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-608721e6-a345-472a-8350-e9c078690299 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:03.253161 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608721e6-a345-472a-8350-e9c078690299 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 02:00:03.255601 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608721e6-a345-472a-8350-e9c078690299 service nova] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 02:00:03.278147 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 700/1379] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 02:00:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 02:00:05.501294 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484c2fb-68c0-4f99-bc29-52a6d98cf8d4 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:05.501852 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484c2fb-68c0-4f99-bc29-52a6d98cf8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:05.501852 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484c2fb-68c0-4f99-bc29-52a6d98cf8d4 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:05.503375 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 680/1380] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 02:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 02:00:05.509311 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777dd2e8-280c-4063-b73d-b2de05f066d0 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:05.509717 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777dd2e8-280c-4063-b73d-b2de05f066d0 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:05.509858 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777dd2e8-280c-4063-b73d-b2de05f066d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:05.521136 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:05.579208 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-777dd2e8-280c-4063-b73d-b2de05f066d0 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 02:00:05.580435 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-777dd2e8-280c-4063-b73d-b2de05f066d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:05.710661 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 701/1381] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 02:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 02:00:05.995652 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484c2fb-68c0-4f99-bc29-52a6d98cf8d4 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 02:00:05.998409 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-f484c2fb-68c0-4f99-bc29-52a6d98cf8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:06.198074 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:06.211611 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484c2fb-68c0-4f99-bc29-52a6d98cf8d4 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 02:00:06.340889 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283385a-ff87-46a2-b886-ff3fbe295a00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f721a3a4eca941989685448a87bb9a19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 02:00:06.341634 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283385a-ff87-46a2-b886-ff3fbe295a00 service nova] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:06.342004 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283385a-ff87-46a2-b886-ff3fbe295a00 service nova] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:06.342340 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283385a-ff87-46a2-b886-ff3fbe295a00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:06.344377 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-3283385a-ff87-46a2-b886-ff3fbe295a00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:06.446412 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283385a-ff87-46a2-b886-ff3fbe295a00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 02:00:06.448047 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283385a-ff87-46a2-b886-ff3fbe295a00 service nova] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 02:00:06.459663 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:06.468817 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 681/1382] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 02:00:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 02:00:08.673889 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7e9ff-14fc-4b8a-958e-269d39daf32f None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:08.674469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7e9ff-14fc-4b8a-958e-269d39daf32f None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:08.674469 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7e9ff-14fc-4b8a-958e-269d39daf32f None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:08.675727 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 702/1383] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 02:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 02:00:08.682546 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14442310-cf07-4ab8-9917-a253c6690965 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:08.683030 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14442310-cf07-4ab8-9917-a253c6690965 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:08.683149 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14442310-cf07-4ab8-9917-a253c6690965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:08.749364 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-14442310-cf07-4ab8-9917-a253c6690965 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 02:00:08.750601 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-14442310-cf07-4ab8-9917-a253c6690965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:08.848914 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 682/1384] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 02:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 02:00:09.135298 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7e9ff-14fc-4b8a-958e-269d39daf32f None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 02:00:09.137268 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-b2f7e9ff-14fc-4b8a-958e-269d39daf32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:09.239933 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:09.314845 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7e9ff-14fc-4b8a-958e-269d39daf32f None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 02:00:09.449725 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db1a35-619d-4802-ba40-791b25f51b11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f721a3a4eca941989685448a87bb9a19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 02:00:09.450361 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db1a35-619d-4802-ba40-791b25f51b11 service nova] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:09.450645 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db1a35-619d-4802-ba40-791b25f51b11 service nova] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:09.450913 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db1a35-619d-4802-ba40-791b25f51b11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:09.452538 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-91db1a35-619d-4802-ba40-791b25f51b11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:09.513122 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:09.554232 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db1a35-619d-4802-ba40-791b25f51b11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 02:00:09.555851 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db1a35-619d-4802-ba40-791b25f51b11 service nova] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 02:00:09.574793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 703/1385] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 02:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 02:00:11.824276 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a635ab-2166-4eed-b24b-b79234d5d507 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:11.824792 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a635ab-2166-4eed-b24b-b79234d5d507 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:11.824792 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a635ab-2166-4eed-b24b-b79234d5d507 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:11.826108 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 683/1386] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 02:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 02:00:11.831789 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c3919-a773-40c4-bd7f-67c0bd2fedf3 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:11.832256 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c3919-a773-40c4-bd7f-67c0bd2fedf3 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:11.832364 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c3919-a773-40c4-bd7f-67c0bd2fedf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:11.901520 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-909c3919-a773-40c4-bd7f-67c0bd2fedf3 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 02:00:11.902969 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-909c3919-a773-40c4-bd7f-67c0bd2fedf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:12.002018 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 704/1387] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 02:00:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 02:00:12.295547 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a635ab-2166-4eed-b24b-b79234d5d507 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 02:00:12.297448 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-18a635ab-2166-4eed-b24b-b79234d5d507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:12.373180 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:12.447146 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a635ab-2166-4eed-b24b-b79234d5d507 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 02:00:12.561603 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f71f9-7c1a-4952-8424-d9d4c3f84a49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f721a3a4eca941989685448a87bb9a19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 02:00:12.562162 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f71f9-7c1a-4952-8424-d9d4c3f84a49 service nova] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:12.562437 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f71f9-7c1a-4952-8424-d9d4c3f84a49 service nova] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:12.562701 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f71f9-7c1a-4952-8424-d9d4c3f84a49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:12.564576 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-e92f71f9-7c1a-4952-8424-d9d4c3f84a49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:12.604758 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:12.655835 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f71f9-7c1a-4952-8424-d9d4c3f84a49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 02:00:12.657431 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f71f9-7c1a-4952-8424-d9d4c3f84a49 service nova] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 02:00:12.675595 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 684/1388] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 02:00:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 02:00:13.535880 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25940bcd-252d-4d67-8d28-c805373fd249 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:13.536467 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25940bcd-252d-4d67-8d28-c805373fd249 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:13.536467 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25940bcd-252d-4d67-8d28-c805373fd249 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:13.538511 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 705/1389] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 02:00:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 02:00:13.543565 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3568cf-204a-4cef-aa1a-dca7c8bf9a3b None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:13.543991 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3568cf-204a-4cef-aa1a-dca7c8bf9a3b None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:13.544145 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3568cf-204a-4cef-aa1a-dca7c8bf9a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:13.613856 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-0b3568cf-204a-4cef-aa1a-dca7c8bf9a3b None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 02:00:13.615145 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-0b3568cf-204a-4cef-aa1a-dca7c8bf9a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:13.712064 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 685/1390] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 02:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 02:00:14.010700 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f681a5d-2b61-462d-9105-d4d498f6c550 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:14.011210 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f681a5d-2b61-462d-9105-d4d498f6c550 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:14.011477 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f681a5d-2b61-462d-9105-d4d498f6c550 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:14.012800 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 706/1391] 10.222.0.15 () {54 vars in 966 bytes} [Wed Sep 14 02:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 02:00:14.018065 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77179341-4ac6-43cb-a194-f98a430d9a70 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:14.018351 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77179341-4ac6-43cb-a194-f98a430d9a70 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:14.018621 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77179341-4ac6-43cb-a194-f98a430d9a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:14.034723 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:14.098452 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-77179341-4ac6-43cb-a194-f98a430d9a70 None None] MFA Rules not processed for user `5a5191afa5f246269d5b4a60972332a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 02:00:14.099660 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-77179341-4ac6-43cb-a194-f98a430d9a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:14.174445 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 686/1392] 10.222.0.15 () {58 vars in 1063 bytes} [Wed Sep 14 02:00:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 02:00:14.183629 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f681a5d-2b61-462d-9105-d4d498f6c550 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 02:00:14.185119 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-1f681a5d-2b61-462d-9105-d4d498f6c550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:14.217283 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:14.290211 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f681a5d-2b61-462d-9105-d4d498f6c550 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 02:00:14.364478 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fccbce7-2014-4a8b-a2bb-4291ca0be7f8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a5191afa5f246269d5b4a60972332a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 02:00:14.365124 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccbce7-2014-4a8b-a2bb-4291ca0be7f8 service glance] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:14.365404 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccbce7-2014-4a8b-a2bb-4291ca0be7f8 service glance] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:14.365677 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccbce7-2014-4a8b-a2bb-4291ca0be7f8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:14.367319 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-8fccbce7-2014-4a8b-a2bb-4291ca0be7f8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:14.454966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:14.488196 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fccbce7-2014-4a8b-a2bb-4291ca0be7f8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 02:00:14.489638 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fccbce7-2014-4a8b-a2bb-4291ca0be7f8 service glance] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 02:00:14.509488 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 707/1393] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 02:00:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 02:00:15.024468 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae34cd-28c7-4f3e-b718-ad59da3def91 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:15.025108 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae34cd-28c7-4f3e-b718-ad59da3def91 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:15.025386 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae34cd-28c7-4f3e-b718-ad59da3def91 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:15.027331 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 687/1394] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 02:00:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 02:00:15.032895 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da26cc-1dae-40f6-8c2d-362dd4b9e245 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:15.033371 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da26cc-1dae-40f6-8c2d-362dd4b9e245 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:15.033458 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da26cc-1dae-40f6-8c2d-362dd4b9e245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:15.118571 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-87da26cc-1dae-40f6-8c2d-362dd4b9e245 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 02:00:15.119840 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-87da26cc-1dae-40f6-8c2d-362dd4b9e245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:15.242025 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 708/1395] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 02:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 02:00:15.312171 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb2ea5-3baa-4d2b-8a2f-a2a468254c8f None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:15.312455 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb2ea5-3baa-4d2b-8a2f-a2a468254c8f None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:15.312721 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb2ea5-3baa-4d2b-8a2f-a2a468254c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:15.387437 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-3bcb2ea5-3baa-4d2b-8a2f-a2a468254c8f None None] MFA Rules not processed for user `e1c8d2eec886482eb135e19ded2c61ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 02:00:15.388649 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-3bcb2ea5-3baa-4d2b-8a2f-a2a468254c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:15.446687 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:15.479666 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 688/1396] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Sep 14 02:00:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 02:00:15.526187 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce9e41-d3a2-4f6d-b9c2-23ab63785fd9 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:15.526482 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce9e41-d3a2-4f6d-b9c2-23ab63785fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:15.526807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce9e41-d3a2-4f6d-b9c2-23ab63785fd9 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:15.528171 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 709/1397] 10.222.0.15 () {54 vars in 943 bytes} [Wed Sep 14 02:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 02:00:15.534536 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2382d-d160-4177-90a6-aa00a39aedd3 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:15.534829 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2382d-d160-4177-90a6-aa00a39aedd3 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:15.535086 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2382d-d160-4177-90a6-aa00a39aedd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:15.554116 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce9e41-d3a2-4f6d-b9c2-23ab63785fd9 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 02:00:15.555719 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-13ce9e41-d3a2-4f6d-b9c2-23ab63785fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:15.596966 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:15.614347 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-66b2382d-d160-4177-90a6-aa00a39aedd3 None None] MFA Rules not processed for user `21f481c1f60043e9b0e25ba49b30715f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 02:00:15.618403 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-66b2382d-d160-4177-90a6-aa00a39aedd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:15.707694 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce9e41-d3a2-4f6d-b9c2-23ab63785fd9 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 02:00:15.728027 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 689/1398] 10.222.0.15 () {58 vars in 1040 bytes} [Wed Sep 14 02:00:15 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 02:00:15.738875 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2382d-d160-4177-90a6-aa00a39aedd3 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 02:00:15.748986 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2382d-d160-4177-90a6-aa00a39aedd3 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 02:00:15.776086 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:15.829545 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:15.831883 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d9f32-d256-4df1-b9fb-d87f125aeee2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f721a3a4eca941989685448a87bb9a19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 02:00:15.832852 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d9f32-d256-4df1-b9fb-d87f125aeee2 service nova] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:15.834113 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d9f32-d256-4df1-b9fb-d87f125aeee2 service nova] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:15.834696 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d9f32-d256-4df1-b9fb-d87f125aeee2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:15.836834 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-597d9f32-d256-4df1-b9fb-d87f125aeee2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:15.858577 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e3d03-b031-4bc2-9656-187114bd0c5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21f481c1f60043e9b0e25ba49b30715f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 02:00:15.859185 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e3d03-b031-4bc2-9656-187114bd0c5d service swift] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:15.859461 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e3d03-b031-4bc2-9656-187114bd0c5d service swift] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:15.859730 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e3d03-b031-4bc2-9656-187114bd0c5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:15.861367 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-d77e3d03-b031-4bc2-9656-187114bd0c5d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:15.931402 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d9f32-d256-4df1-b9fb-d87f125aeee2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 02:00:15.933213 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d9f32-d256-4df1-b9fb-d87f125aeee2 service nova] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 02:00:15.950881 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e3d03-b031-4bc2-9656-187114bd0c5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 02:00:15.951826 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 710/1399] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 02:00:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 02:00:15.952502 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e3d03-b031-4bc2-9656-187114bd0c5d service swift] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 02:00:15.961084 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 690/1400] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Sep 14 02:00:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 02:00:18.562793 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddb299-b091-4679-9ad4-deedd2057574 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:18.563334 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddb299-b091-4679-9ad4-deedd2057574 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:18.563334 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddb299-b091-4679-9ad4-deedd2057574 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:18.564664 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 711/1401] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 02:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 02:00:18.570670 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96981f9-01b6-4e06-ac9e-d3609fdfffa1 None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:18.571087 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96981f9-01b6-4e06-ac9e-d3609fdfffa1 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:18.571279 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96981f9-01b6-4e06-ac9e-d3609fdfffa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:18.649290 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-b96981f9-01b6-4e06-ac9e-d3609fdfffa1 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 02:00:18.650452 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-b96981f9-01b6-4e06-ac9e-d3609fdfffa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:18.657462 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:18.758029 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 691/1402] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 02:00:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 02:00:19.058562 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddb299-b091-4679-9ad4-deedd2057574 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 02:00:19.060276 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-4cddb299-b091-4679-9ad4-deedd2057574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:19.200415 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:19.232001 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddb299-b091-4679-9ad4-deedd2057574 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 02:00:19.365807 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2328ad-adfb-4c49-88dc-bab3d8dfe257 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f721a3a4eca941989685448a87bb9a19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 02:00:19.366440 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2328ad-adfb-4c49-88dc-bab3d8dfe257 service nova] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:19.366714 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2328ad-adfb-4c49-88dc-bab3d8dfe257 service nova] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:19.366990 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2328ad-adfb-4c49-88dc-bab3d8dfe257 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:19.368580 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-cd2328ad-adfb-4c49-88dc-bab3d8dfe257 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:19.459613 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:19.470673 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2328ad-adfb-4c49-88dc-bab3d8dfe257 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 02:00:19.472358 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2328ad-adfb-4c49-88dc-bab3d8dfe257 service nova] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 02:00:19.492559 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 712/1403] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 02:00:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 02:00:21.725312 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df3efe-6a75-441b-91a7-f7ba20883384 None None] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:21.725860 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df3efe-6a75-441b-91a7-f7ba20883384 None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:21.725860 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df3efe-6a75-441b-91a7-f7ba20883384 None None] PATH_INFO: `/` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:21.727413 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 692/1404] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 02:00:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 02:00:21.732384 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b0485-38e6-488d-843a-dbe3f06010e1 None None] REQUEST_METHOD: `POST` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:21.732840 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b0485-38e6-488d-843a-dbe3f06010e1 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:21.732959 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b0485-38e6-488d-843a-dbe3f06010e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:21.803064 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.auth.core [None req-d62b0485-38e6-488d-843a-dbe3f06010e1 None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 02:00:21.806067 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-d62b0485-38e6-488d-843a-dbe3f06010e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:21.914317 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 713/1405] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 02:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 02:00:22.211974 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df3efe-6a75-441b-91a7-f7ba20883384 None None] Authenticating user token {{(pid=81383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 02:00:22.213550 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-53df3efe-6a75-441b-91a7-f7ba20883384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:22.362793 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:22.375452 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df3efe-6a75-441b-91a7-f7ba20883384 None None] Validating token access rules against request {{(pid=81383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 02:00:22.507389 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e7c05-3a60-4e65-bb5f-f38481e849b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f721a3a4eca941989685448a87bb9a19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 02:00:22.507990 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e7c05-3a60-4e65-bb5f-f38481e849b3 service nova] REQUEST_METHOD: `GET` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:22.508272 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e7c05-3a60-4e65-bb5f-f38481e849b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:22.508540 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e7c05-3a60-4e65-bb5f-f38481e849b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:22.510196 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-0a4e7c05-3a60-4e65-bb5f-f38481e849b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:22.609708 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e7c05-3a60-4e65-bb5f-f38481e849b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 02:00:22.611704 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e7c05-3a60-4e65-bb5f-f38481e849b3 service nova] RBAC: Authorization granted {{(pid=81383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 02:00:22.625520 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=100 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:22.637559 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 693/1406] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 02:00:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 02:00:24.850030 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac487a1-a6c3-4442-b434-106b67f438e0 None None] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:24.850617 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac487a1-a6c3-4442-b434-106b67f438e0 None None] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:24.850617 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac487a1-a6c3-4442-b434-106b67f438e0 None None] PATH_INFO: `/` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:24.855741 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 714/1407] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 14 02:00:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 02:00:24.857958 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4d1c8-0b1f-473b-9bc4-3898d19d04de None None] REQUEST_METHOD: `POST` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:24.858437 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4d1c8-0b1f-473b-9bc4-3898d19d04de None None] SCRIPT_NAME: `/identity` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:24.858646 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4d1c8-0b1f-473b-9bc4-3898d19d04de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:24.935378 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.auth.core [None req-4be4d1c8-0b1f-473b-9bc4-3898d19d04de None None] MFA Rules not processed for user `25374d78618f4e32be8780901a31679f`. Rule list: `[]` (Enabled: `True`). {{(pid=81383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 02:00:24.936680 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG keystone.common.fernet_utils [None req-4be4d1c8-0b1f-473b-9bc4-3898d19d04de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:25.056347 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: [pid: 81383|app: 0|req: 694/1408] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 14 02:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 02:00:25.349311 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac487a1-a6c3-4442-b434-106b67f438e0 None None] Authenticating user token {{(pid=81382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 02:00:25.351018 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-dac487a1-a6c3-4442-b434-106b67f438e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:25.430159 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:25.520646 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac487a1-a6c3-4442-b434-106b67f438e0 None None] Validating token access rules against request {{(pid=81382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 02:00:25.646841 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00464e19-298e-4f85-8f8e-bc09e2a2cdb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f721a3a4eca941989685448a87bb9a19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e04b16dba1c4de79dc6b1a5e16ed907', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 02:00:25.647451 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00464e19-298e-4f85-8f8e-bc09e2a2cdb5 service nova] REQUEST_METHOD: `GET` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 02:00:25.647730 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00464e19-298e-4f85-8f8e-bc09e2a2cdb5 service nova] SCRIPT_NAME: `/identity` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 02:00:25.648004 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00464e19-298e-4f85-8f8e-bc09e2a2cdb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 02:00:25.649628 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.fernet_utils [None req-00464e19-298e-4f85-8f8e-bc09e2a2cdb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 02:00:25.674795 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=100 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:25.753197 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00464e19-298e-4f85-8f8e-bc09e2a2cdb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 02:00:25.755484 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00464e19-298e-4f85-8f8e-bc09e2a2cdb5 service nova] RBAC: Authorization granted {{(pid=81382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 02:00:25.775261 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: [pid: 81382|app: 0|req: 715/1409] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 14 02:00:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 02:00:35.053202 dv-cinder-857075-5-iscsi devstack@keystone.service[81383]: DEBUG dbcounter [-] [81383] Writing DB stats keystone:SELECT=67 {{(pid=81383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 02:00:35.772643 dv-cinder-857075-5-iscsi devstack@keystone.service[81382]: DEBUG dbcounter [-] [81382] Writing DB stats keystone:SELECT=35 {{(pid=81382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}